Building Africa’s AI Initiative
Considerations behind the largest and most complex undertaking of its kind in Africa’s history.
About
In collaboration with academia, Cirrus is a private sector led initiative bringing together academia and industry for the establishment of a world class AI research and application capability for Africa. Although launched in 2019, work on Cirrus started several years earlier which included laying the ground work for Wits University to host the Cirrus infrastructure.
History
Cirrus was conceived over a decade ago but work on the first formal proposal started in 2017. At the time, Dr Dean Barrett, who works at the Brazilian Synchrotron Light Laboratory (LNLS), was seeking a way to apply machine learning to the vast amount of data they were producing.
Working with his fellow collaborator at Wits University, Dr Roy Forbes, it soon became apparent that Wits had similar needs and should be incorporated into the requirement. This was detailed in a proposal document and in early 2018 presented to Wits.
Following meetings with stakeholders from across the university, Professor Dean Brady, Head of the School of Chemistry and Director of the Molecular Science Institute, with the support of Professor Zeblon Vilakazi, took the lead on pushing Cirrus forward within Wits with the intention that this should benefit not only Wits University but all academic and research institutions in Africa.
AI transforms science
Before getting into some of the Cirrus specific details, I am going to take a step back and confirm why so much time and money has been spent on this effort and answer why what is being done is so important.
Firstly, machine learning is at the core of a technological and societal artificial intelligence revolution that is impacting numerous areas of society, including scientific research.
Secondly, Africa is absent from this revolution and has not made investments and undertaken initiatives that can compare with those in North America, UK/Europe and more recently in Asia.
Thirdly, academic institutions are struggling as a significant part of the basic research is now being done in industry with rapid commercialization of results and there is now little distinction between academic research and industrial labs.
For those concerned with scientific research and the impact of AI, the report by Rick Stevens and Valorie Taylor from Argonne National Lab is highly recommended.
Materials science example
As an example of how materials research has been transformed, the days of researchers dressed in white coats synthesizing materials is rapidly coming to an end.
Modern-day efforts now deploy fully automated processes for synthesis, characterisation and design. While automated and heavily reliant on current AI techniques, this is still a human-in-the-loop science.
This human-in-the loop approach means Intelligent Augmentation as referred to by Michael Jordan from Berkeley is perhaps the most apt description of the current state of affairs in AI.
Plans in the United States
Another example worth noting, in the United States the White House has recently announced plans to invest 1 billion dollars to advance AI across several industries. The effort spans a number of universities and fields.
On the African side
On the African side, things however are a little different. Specifically, there is no AI ecosystem to support AI research and its application across various scientific fields.
I have decomposed the AI ecosystem into 5 key components:
- The engineering pipeline which includes hardware, software, machine learning and data engineers
- Computing infrastructure
- Data infrastructure
- And industrial research operations and
- Corporate Venture Capital operations
Importantly, these components are not present in Africa and I will touch on some of these challenges in the next few paragraphs. The absence of these components makes AI research and its application across various scientific fields for the benefit of academia and industry in the region a nonstarter.
Matters engineering
On the engineering side, there is significant technical debt underpinning machine learning techniques and systems that constitute what is commonly referred to as AI. The technical debt is comprised not only of engineering expertise, but also data, hardware and software.
This technical debt is a substantial roadblock to various scientific researchers wishing to apply AI to their work.
Computing infrastructure and energy
Looking at some of the major AI systems over the years in the context of the compute required. If we compare AlexNet to AlphaGoZero the price of progress was a 300 000x increase in the compute used. This is a doubling period every 3.5 months. For comparison, Moore’s Law had an 18-month doubling period
And it is not just being able to leverage computing that is necessary but knowing how to use it optimally. The graphic below shows the 62 000x performance improvement for matrix multiplication when using SIMD compared to native python. (Note that the Y-axis is logarithmic.)
SIMD standing for Single Instruction Multiple Data.
Such techniques/implementations are known and practiced by seasoned engineers but are not always apparent to typical nonspecialist persons in industry and academia that are looking to leverage AI in their work.
The takeaway here is that without the necessary computing infrastructure and the requisite expertise to leverage it, the results are slow execution and ultimately slow science.
To underscore the extent of the impact, referenced is a 2020 paper that details almost a 2 billion times acceleration of scientific simulation by using a technique called deep neural architecture search.
Researchers in Africa cannot afford a delta of 2 billion x when undertaking their scientific simulations.
Computing infrastructure and cost
These AI models and their use of infrastructure come at a pretty penny. For example Google’s Meena chatbot released earlier this year cost around 1,5 million dollars to train.
But that is nothing compared to OpenAI’s GPT-3 which likely cost somewhere in the region of 10 million dollars to train.
The cloud
But don’t worry “we can just use the cloud” some say. There are some not so small problems with that though. For a start there is no High Performance Computing research cloud for AI in Africa. Not even the United States has an HPC research cloud for AI, although there are efforts underway to build one. It should be noted that those plans include a budget of 7 billion dollars a year.
Further, building such a cloud entails confronting the same challenges that are being addressed as part of building Cirrus, with the addition of a few more.
Most importantly, the existence of an HPC research cloud does not address the problem of providing user support. An HPC cloud for research is great for those researchers experienced in using such technology. Certainly, most researchers do not have such experience.
There is also a general lack of cloud related research work in Africa. Such research includes developing new cloud related:
- operating systems
- virtualization methods
- performance variability studies
- power management research
- software defined networking
- artificial intelligence
- and resource management
Testbeds such as Chameleon Cloud offer the large-scale deeply reconfigurable experimental platforms to support such research and run into the tens of millions of dollars to establish and maintain.
As you can see there are no users from Africa and this is not because African users have their own testbed.
AI focused hardware
In terms of AI focused hardware, what is it and what does it look like? Below are listed some of the major and emerging players:
- NIVIDA
- Groq
- Cerebras
- Preferred Networks
- Graphcore
- SambaNova
- Habana (now part of Intel)
- Lightmatter
The Youtube links listed above are to presentations from the respective providers from the 2019 ISC HPC conference, while the link to the Lightmatter presentation is from Hot Chips 32.
As a side note, the evaluation of AI focused hardware performance does not follow traditional TOPS per watt. TOPS standing for Tera-Operations Per Second per watt, and has resulted in the new evaluation benchmarks, some of which are well covered in this article.
On the data front
Moving onto matters on the data front. To advance research in Africa there is a need for a data commons. In the simplest of terms a data commons is the “place” where data lives.
Over the years there have been countless presentations and articles on how data is now the new “black gold” of the modern economy. Black gold being the reference to crude oil. Whether or not this is true is another matter, but just like crude oil needs to reside in a storage facility prior to downstream processing, so to does data. This could be the results of a clinical trial, hourly readings of ocean temperature, tracking data from migratory birds, satellite imagery, data from a x-ray diffractometer, or any other scientific instrument.
While at the lowest level the infrastructure of a data commons is constituted of hardware and software, at a higher level it also drives the implementation of a systematic data strategy that amongst other things ensures that investments in data generating infrastructure, like scientific research infrastructure and instrumentation, is fully realised.
Several universities in Africa have scientific instrumentation running into the high tens of millions of dollars, yet the data generated by such infrastructure is going into a black hole. The black hole being researcher hard drives, thumb drives, as well as personal and institutional cloud storage folders etc. Ten years from now you can be sure that most of this data will be difficult locate and much of it will likely have disappeared.
Clearly this situation is unacceptable for modern day scientific research and there are many efforts around the world to address this problem. A good example of which is ARDC — the Australian Research Data Commons.
ARDC connects research infrastructure from universities and research institutions from across Australia and New Zealand to a data commons that supports multiple scientific domains. Examples include the Characterisation, Bio and Eco Commons, with further details on these listed at the link. Another example is the Cancer Research Data Commons, which is part of the National Cancer Institute in the US.
Clearly, having data strewn across various institutions and regions does nothing to solve the problem and only creates more data silos, unless there is connectivity between them that supports collaboration and data sharing. Ensuring connectivity between these platforms is possible and is something that Cirrus is presently exploring. Ideally what we are working towards is a system that supports the machine learning and data ecosystem. At present such a system only exists at but a handful of the world’s largest technology companies.
For example, the MLdp system depicted comes from a 2019 paper from a team at Apple.
Connectivity
Another challenge in Africa that needs to be dealt with is connectivity. Think about our black gold analogy. The oil field (our data generators) needs to be connected to the storage facility (our data management platform come data commons). The storage facility then needs to connect to the processing facility (the high performance computing platform).
At present the connectivity in Africa is abysmal. If Africa is going to participate in coming exascale scientific work then plans needs to be formulated and implemented to support such. These plans should at a minimum support 1 terabit regional supercore and 100 gigabit edge nodes. So for example Wits, the University of Johannesburg and the University of Cape Town would connect to each other on the supercore, while Wits would connect to the LNLS for example, through the edge node.
That this is a matter of importance is one of the major reasons why Cirrus is seeking to host an edge node as part of FABRIC.
FABRIC is a testbed for the development of the next generation internet architecture. This will permit researchers from Africa to participate in the development of next generation internet technologies.
Worth noting is the new Facebook project (2Africa) which will see the installation of a subsea cable capable of supporting more traffic than all the existing cables combined.
Research and Corporate Venture Capital (CVC)
Moving onto industry and corporate research and corporate venture capital operations, in Africa these operations are near non-existent. I pulled the examples of two firms that readers might be familiar with and that are operational on the continent, Toyota and Microsoft.
Toyota’s research operation, Toyota Research Institute (TRI) and its AI corporate venture capital operation, Toyota AI Ventures, at the time of this writing have no operations on the continent. Microsoft’s research operation, Microsoft Research, and its AI corporate venture capital operation, M12 are also missing in action in Africa. To be clear, I am not trying to single out these two firms, they are simply two examples from familiar organisations.
If you had to go through a list of the world’s leading publicly listed multinationals, taking your pick from the Nasdaq, Dow, S&P, FTSE, Xetra DAX etc, you will find this to be true in more than 90% of the instances, and it’s not as if Africa can continue to be absent from this landscape. These firms are major players in the research and commercialisation ecosystem.
TRI for example, is a leader in materials science research and has recently announced efforts to establish another corporate venture capital operation, Woven Capital, with 800 million dollars in investment.
The structure and domicile of these operations warrants consideration and some brief discussion.
Structures to fund hardtech innovation
Below is an example of a structure developed by the U.S. Department of Energy to support hardtech funding and commercialisation. This particular structure has been called a first look funding structure.
The reason I have brought this up is not to delve into the specific details of this particular funding structure, but to point out that these structures are meaningful and serve a specific purpose. It will not be possible for a university in South Africa for example, to simply setup a wholly owned institute or private company and expect such to suffice for effective research and commercialisation purposes with industry. I will be revisiting this point in the Cirrus context a little later on.
One example supporting successful industry and academic collaboration on research and commercialisation is Semiconductor Research Corporation. Over at least three decades of operation SRC has undertaken over 3700 projects, provided over 2 billion dollars in funding, and generated over 700 patents. Importantly, SRC has amongst other things a clear intellectual property strategy that supports industry participation.
Intellectual Property (IP)
If our list of challenges was not long enough, you can add intellectual property to it — and its an important one.
The Intellectual Property Rights Act (IPR Act) in South Africa determines that any invention resulting from research contracted by an outside party with a university/research council will belong to the university/research council unless the outside party pays for such research based on a full-cost model. While this sounds inviting and easy, a full-cost model is very difficult to determine, and auditors can and do argue the details for years.
So, this approach is fraught with pitfalls and is one of the major reasons why several of the regions largest corporations have taken all their previously local research off-shore.
Some possible considerations for dealing with this problem include:
1) Avoidance: which entails avoiding university/research councils altogether.
2) Forbearance: under this approach there would be joint ownership with university commercial forbearance.
Under forbearance, IP solely invented or authored by startup employees would be jointly owned by the startup and the university. However, the university agrees to forbear from commercially licensing the IP, and can only use the IP for research, education and non-profit purposes. This approach is used by the Universality of California at Berkeley in their Shared Special User Facility for Innovation & Entrepreneurship (SSUFIE) program.
3) The third and final approach would be to “box it”.
This would entail executing some sort of standardised and clearly defined and predetermined full cost model, thus reducing ex-post pitfalls like audits.
Cirrus overview
Now that I have laid the groundwork on what we are dealing with, lets jump into the details around Cirrus and how these and other matters are dealt with. Cirrus has three major components:
1) Cirrus itself houses the cooperation programs, the state-of-the-art computing infrastructure and the open learning programs.
2) The Cirrus FOUNDRY is equipped with everything needed to bridge what is called the “Valley of Death”, and that is overcoming the challenge of turning a start-up idea or scientific research into large-scale commercial application. Simply, the business of the Cirrus FOUNDRY is building other businesses.
The Cirrus FOUNDRY is characterised by:
- ideation generally carried out by the in-house team
- the use of in-house staff in product creation
- building multiple products centered around AI
- with the objective of being standalone businesses
- retaining some ownership stake and has a capital fund
For these reasons it is not an incubator or accelerator and referred to as a foundry.
3) The third component, the Cirrus FOUNDRY Fund is the in-house fund to support start-ups in the Cirrus FOUNDRY to ensure they are not wholly dependent on outside capital.
The Cirrus FOUNDRY Fund has a target capitalization of 35 million dollars and will undertake pre-seed and seed stage investments. For most investments, capital will be allocated at the pre-seed stage with investments of up to 3.7 million South African rand. The FOUNDRY Fund will also follow a lead investor in seed rounds, with investments of up to 7.5 million South African rand.
Context in the R&D landscape
In the simplest of terms Cirrus is focused on the research stage. The R in and R&D. The Cirrus FOUNDRY is focused on the D in R&D. Thus any development, deployment and possible dollars are the concern of the Cirrus FOUNDRY.
The Cirrus FOUNDRY occupies a strategic position.
1) Research
On the research front the Cirrus FOUNDRY will concern itself with knowing what facilities and programs exist and where and how to leverage them.
2) Innovation
On the innovation front, the Cirrus FOUNDRY will be in a position of knowing what innovations lie in the pipeline and what industries are most likely to be impacted.
3) Industry
On the industrial front, the Cirrus FOUNDRY will be in a position of knowing what industry is looking for and importantly, what will constitute technical validation thus creating successful technology pathways.
Host university
Wits University is working to be the host university for Cirrus. This means that the Cirrus infrastructure will be based at Wits University and Wits will lead the formation of the Cirrus Consortium. On the Wits side, Vice Chancellor, Professor Zeblon Vilakazi, is the project owner and Professor Emeritus Barry Dwolatzky is the Cirrus Project Lead.
A multiyear process
Establishing Cirrus has been a multiyear process.That something like this has never been done before on the African continent has made the process even more challenging.
The first step in the process was to lay the foundation which included agreements between Cirrus and the host institution, including the collaboration and land agreement. In addition, the host institution is also a party on the Cirrus Advisory Council and the Cirrus Consortium.
The second major step in the process is building the use case and takes us to where we are today. The use case is the justification for the necessary investment. You will note strong parallels between this and the call to action on the United States research cloud.
The third and final stage in the process is industry engagement where the funding is secured. This effort will be led by a global investment bank. The Cirrus budget is 200 million dollars and the amount needs to be placed into context. While this might sound like a lot for Africa, and it is, it is small in comparison to efforts in North America, the UK, Europe and Asia. Less than this and Cirrus is simply not feasible, more than this and we start to exceed the capacity of what is needed and possible in Africa.
Private and public sector engagement
Cirrus is structured to support public and private sector engagement. Private sector organisations supporting Cirrus are referred to as Strategic Founding Partners (SFP’s). All Strategic Founding Partners receive equity in Cirrus, a seat on the Cirrus Advisory Council and permanent admission to the Cirrus Partner program.
Academic and research institutions access Cirrus infrastructure and resources through the Cirrus Consortium.
In terms of the role of government, the Cirrus approach is for government to support its local academic and research institutions. This approach ensures that Cirrus does not take away from or compete for the scarce resources needed by academic and research institutions. Further and very importantly, this approach ensures that there is no bureaucratic, political and poor governance spill over into Cirrus.
The engagement map can be divided up into three components:
1) The funding which comes from the private sector.
2) The users and major beneficiaries from the academic and research institutions, which provide the use case.
3) The private sector programs which include technology transfer/sharing strategies to support research and commercialisation.
The use case
The use case answers the questions of who is going to use Cirrus, what will they use it for, and how this will transform the continent. It is primarily for the users of Cirrus to answer this question, that is the academic and research institutions.
There are some ancillary challenges that enter into the discussion here. Specifically, the need to address:
1) Intra and inter Africa connectivity
2) and a data management platform come data commons to support science across the continent
At present there is a lack of concrete plans from academic and research institutions on the continent tackling these two challenges. While the implementation of a data management platform, the establishment of a data commons and improving connectivity are not the primary objectives of Cirrus, we should be well positioned to tackle these important issues and we have several ideas in the pipeline.
Cirrus Consortium
Turning attention to the Cirrus Consortium. As the host university, Wits University will be leading the engagement with academic and research institutions wishing to join the Consortium.
The table below provides a summary of what Consortium Members will have access to. It covers Cirrus, the Cirrus FOUNDRY and the Cirrus FOUNDRY Fund.
Importantly, there are no Consortium Membership dues, so participation in the Consortium is free. Thus no academic and research institutions are excluded because of financial constraints and the extent of the benefits they accrue depends only on the extent to which they participate.
Above is a schematic view of the Cirrus Consortium structure. To simplify the Consortium documentation the documents including the By-laws, Charter, Membership Benefits and the Consortium Membership Agreement are each a separate document. Consortium Members need only process the Consortium Membership Agreement which is a four-page document.
The Consortium Membership Agreement supports two classes of membership — Tier one and Tier two. Tier one is differentiated from tier two by a higher level of engagement through the appointment of an Ambassador. The Ambassadors serve as community builders, making connections between people and resources and will be senior strategic individuals within their local institution and will typically be supported by a project manager. Collectively the Ambassadors forms part of the Ambassador Advisory Network which provides input and direction to the Consortium governing body, the Cirrus Advisory Council.
You will recall that I spoke about the LNLS at the beginning of the lecture when I covered the origins of Cirrus. The LNLS is Brazil’s new fourth generation synchrotron and is Brazil’s largest and most complex technology project. It is also provides a stellar use case for Cirrus.
If you look at the worlds most powerful computing platforms, most are used alongside such operations. For example, the worlds most powerful computing platform, Fugaku, is installed at the RIKEN Center for Computational Science in Kobe, Japan and supports amongst other things the Riken SPring-8 synchrotron facility.
Funding
The current list of prospective SFP’s stands at 154 firms and this does not include foundations and philanthropies. It is worth noting that NONE of these firms are headquartered or domiciled in Africa. It is our intention to secure the participation of approximately 10 to 15% of these SFP’s. That translates to 10–25 SFP’s contributing between 7 to 20 million USD in funding each.
Importantly, funding only is not possible and each SFP is required to include the participation of their research and CVC operations. Amongst other things this participation enables later stage funding for technologies developed on the continent and access to critical markets, which is particularly relevant in the hardtech space.
Placing the economics into context, keep in mind that the GDP of South Africa is roughly equivalent to that of the state of Tennessee. Africa’s GDP as a whole is roughly equal to that of France. Thus, waving the African market as a carrot to entice the participation of international firms, alone, has little going for it and there needs to be more.
Most SFP’s for example, know very little about any university in the region and thus the focal point when engaging with these firms is on the use case discussed earlier — supporting an entire region, not merely a single university, nor group of universities, nor a single country.
Governments in the region are also not able to build Cirrus. For a start, governments in the region do not have the funding available, nor the expertise. Attempts at some sort of public private partnership are also not feasible as it raises unresolved intellectual property issues and still does not address what government contribution would be. As mentioned earlier, to the extent that government has any funding available, either now or in the future, this should be directed towards supporting existing academic and research institutions. On top of this, consideration must also be given to dealing with the connectivity and data management challenges and how that can be supported.
Pressures on the academic front
The ongoing pressures for universities to contain costs and change the way they fund themselves, places emphasis on two points:
1) That the current university models are not sustainable, underscores that universities themselves will not be able to bring the capabilities discussed under Cirrus into existence, but rather need to act as important stakeholders in such an effort.
2) Disciplined technology programs are needed to accelerate the transfer of knowledge into social and economic impacts.
Technology transfer/sharing mechanisms
The Cirrus Partner, Affiliate and Co-development programs support collaboration with industry and contain the necessary provisions for technology transfer and sharing. The table below provides a summary of the Cirrus programs and the associated technology transfer and sharing mechanisms.
As Cirrus is funded and owned by industry, intellectual property originating from Cirrus is not encumbered by the Intellectual Property Rights from Publicly Financed Research and Development in South Africa.
You will note that concerning the programs listed in the above table that the “Digital Asset Locker for universities”, and the “Cirrus FOUNDRY”, there is reference to the IPR Act in South Africa. This is reference to those instances where an invention involves a South African university. For example, a university invention may be brought to Cirrus for further development, or there is the use of university infrastructure and resources in the development of the invention.
In such instances the onus is on the university to deal with the intellectual property matters that deal with the previously mentioned challenges — possibly deploying the strategies mentioned earlier (like forbearance), in order that the resulting invention is attractive for commercialisation.
Corporate structure considerations
The equity in Cirrus is owned by the Strategic Founding Partners funding it. Employees also have a stake through an employee stock ownership program. The employee stock ownership program provides employees with skin-in-the-game which is by far the most effective corporate governance tool.
The holding company cannot be domiciled in South Africa. Some of the factors impacting the domicile of the holding company includes:
- exchange controls
- taxation — including double taxation agreements known as DTA’s
- disclosures — including common reporting standards
- investment protection agreements
- and costs
For these reasons places like Delaware in the United States, Ireland and Luxembourg are some of the commonly used locations. Closer to home the nearest and likely cheapest option is Mauritius. I think it is worth mentioning that just maintaining a corporation structure domiciled in Mauritius incurs a cost running into several hundred thousand South African rand a year.
Given the size of Cirrus FOUNDRY Fund with a target capitalization of 35 million dollars, costs are a significant consideration. For the new Toyota CVC fund mentioned earlier, at 800 million dollars, such costs would be less of a concern for example.
Some targets and constraints
HPC, energy generation and storage, grid and cooling:
Moving over to some of the Cirrus targets and constraints. A top50 position on the top500 list is targeted for the high performance computing platform. This would result in the HPC platform being the most powerful in Africa by a large margin as at the time of writing there is not a single installation from Africa on the list.
To house this platform a state of the art 7000 square foot facility will be built. This will be powered by 178 000 square feet of solar with at least a 30% cell efficiency at the time of installation. Energy storage will consist of at least 14 megawatt hours of storage with at least 75% efficiency.
The grid will be High Voltage Direct Current (HVDC) and there will be no connection to the local AC grid for the generation, storage and HPC platform (see HVDC example in note 1). There will also be no fossil fuel generators. The HPC platform will be power capped, meaning that it is power supply constrained, using what is generated and stored (see power management example in note 2).
The HPC platform will incorporate liquid cooling (see examples in notes 3,4 and 5). At present only some of the compute hardware providers support liquid cooling, however at the time of commissioning we expect most if not all, would have moved to support such which is significantly more efficient.
Academic programs:
Cirrus will have four academic programs:
- Residency
- Intern
- Postdoc
- and Assistantship
The Cirrus FOUNDRY will have its own version of the Intern and Residency programs.
Use of teams
Making all this work, both in terms of establishing Cirrus and when operational, is the use of teams.
Scientists working in a particular domain, perhaps it is an ornithologist, or materials scientist, or marine scientist, he or she is seldom an expert in hardware engineering, software engineering, data engineering, or a machine learning engineer. At the same time hardware, software, data and machine learning engineers are seldom ornithologists, materials scientists or marine scientists.
To successfully apply machine learning to these fields requires people working together in teams. It sounds obvious but to get proficient teams together to support researchers is a rarity — but it is something that Cirrus is built around and will provide to the research community in Africa.
Conclusion
To conclude, many years of work have setup Cirrus as the opportunity to utilise AI to transform academic and industrial research and commercialisation in Africa. Never before has such an effort spanning academia and industry been undertaken on such a scale on the continent. For academia to benefit little more is required than joining the Cirrus Consortium. For government to benefit little more is required than continued support for local academic and research institutions. Working together exciting opportunities await.
Notes
[1] Example: HVDC Power Supply System
[2] Example: Power Management
[3] Example: Cold plate
[4] Example: Cooling Performance Testing of Attaway’s Negative Pressure CDU
[5] Example: Thermosyphon Cooler
