Questions About Development ICTs
My first year at the University of Toronto researching information technologies in international development raised some questions that I still haven't answered.
I am a Ph.D. student at U of T's Faculty of Information Studies and Knowledge Media Design Institute. I have also worked for many years at the International Development Research Centre in Ottawa, first working on a booming Web site, and now coordinating IT strategy and projects. During my first year at FIS exploring the theory and practise of IT projects in development. I tried to take a close look at the research conducted on a prominent and well-regarded information and communications project in Southeast Asia, one that has been touted by UNESCO and the International Telecommunication Union as a success. The objective of the project, like so many telecentre initiatives, had been to bring Internet connectivity to an isolated rural area, in this case using satellite technology.
I had hoped this research would help me in a couple of ways:
First, I wanted to become more familiar with the details of a telecentre project on which significant research had been completed. Why had the project been conceived? What did it accomplish?
Second, I wanted to learn about how telecentres are evaluated. To do this, I planned to compare the evaluation research done at the project to idealized concepts of such research, including the empowerment evaluation suggested by activist James Taylor. My hope was that the deficiencies of the research methodologies used for the project might support that outcomes-based evaluation research (IDRC's chosen method) was a more appropriate and useful approach.
I had just completed a paper comparing the differing knowledge paradigms manifested in the Open Knowledge Network project and the World Bank’s Development Gateway initiative. For some reason I had expected to find in those paradigms a clear demonstration of the practical advantages of decentralized knowledge-sharing models over centralized ones. Instead, I read of a five-year debate between the World Bank and various civil society groups concerning who should control certain bundles of development information, a debate that was at least as ideological as it was practical. It was clear from my research on the OKN that the project had been constructed to ideologically oppose the World Bank’s Gateway. At that point, many of long-standing fears about the nature of international development came resurfaced.
The number of published papers on the Southeast Asian project I had chosen was more than manageable: a dozen peer-reviewed papers on the project proper (mostly given at conferences), less than half that on the evaluation itself from project team members, and a few project reports on the IDRC public Web. Although I might have been able to access some internal IDRC documents concerning the project, I chose not to use them to enhance my research, since such sources are impossible for outside researchers to consult, and the bulk of material on the project was publicly available.
After reviewing the project literature I was left with a feeling of unease that, at least initially, puzzled me. I was surprised that all of the research had been done by the project founders and staff, and that reports of the project’s successes had been written by the agencies that had funded the project in the first place.
Turning to the substance of the project, I found it difficult to identify the "success" that UNESCO and ITU have been promoting. The project studies were all examples of attempted single group, pretest-posttest research. The research team did conduct a series of baseline studies, primarily around expectations and perceived needs. It is difficult to describe this aspect of the research as rigorous: the initial study, conducted through interviews, did not focus on expected changes in the local community or in user behaviour. Worst of all, the researchers did not return to this baseline data to determine what changes the project had brought about, and if expectations had been met. In fact, the final project reports contain information of the most general kind -- 20 people trained, 15 PCs installed, and so on -- making it impossible to determine what had actually changed in the community.
If the objective of the project was to build infrastructure for a remote rural community, the project could be considered successful. However, the history of development is filled with transportation and communications systems being built in the South that are either inappropriate (not responding to real needs), not sustainable (too expensive to maintain), or both. Did the project in fact use appropriate technology?
Because of my position at IDRC as the facilitator of the Centre’s information technology strategic planning, I have regular contact with many individuals responsible for the funding of Centre research projects, including the project in question. I had avoided speaking to any of them about it, but after I had completed my research, I met with several staff people active in IDRC’s Asian ICT efforts. Speaking with them made it clear that important information about the project had not been communicated in the researchers’ papers.
I was told first of all that although the project researchers had written authoritatively about the process of establishing a satellite link for Internet access, the link had not actually been put in place during the funded project period. Furthermore, it was indicated that this was because the project leader had behaved in a culturally insensitive manner and had offended regional telecom bureaucrats, who delayed permission for the satellite installation for more than a year. In fact, I was told the dish was only installed the week following the departure of the research leader from the site, and at the end of his involvement with the project. Training on Internet skills, such as email and Web browsing, had to be done using CD-ROM technology, rather than with the Internet itself.
Most interesting was that, rather than using the computers made available through the project, villagers used the project facilities mainly to telephone relatives in other remote communities and in the country’s largest city, where many of them had gone to work. Again, this went unreported in the published research, which focused on the establishment and use of Internet technologies in the village.
It is understandable, though disappointing, that cultural conflicts around the project were not touched on in any of the papers. This has long been a challenge in international development, and mentioning it even obliquely would have reminded researchers and practitioners that it is still a factor that must be dealt with. More difficult to rationalize, as I read the reports, was the researchers’ emphasis on Internet communications when the technology was not in place. The revelation that a more appropriate communications technology, the telephone, was of greater use to the community casts the project in an unflattering light.
The project had been presented and funded as a research project to explore questions around ICT use in remote rural areas of the South, but a study of the literature, and talks with some of the people who funded it, revealed that the project was not conceived primarily as a research project, but as a development intervention. That is, the project was seen by both funders and researchers first and foremost as a transformative ICT project, designed to have a positive impact on the lives of the villagers by developing local infrastructure and IT skills. Research outputs were considered of secondary importance, existing to justify funding distributed to researchers.
There are obvious and inherent conflicts between the needs of research and the needs of development. Obviously, bias is an issue in any social sciences research, but is a particular challenge in development, where researchers often have a significant stake in the perceived success of a project. Research demands not objectivity, but a detachment that will allow sometimes uncomfortable and challenging observations to be made, a position that I do not believe was possible with the project researchers.
The fact that the project is in development circles considered successful is quite troubling, not because it was an obvious failure, but because we really have no way of knowing, from reading the project literature, if it was successful. We can accept that something may have been learned by doing the project -- how to set up an Internet satellite connection, perhaps -- but the literature presents amazingly scant evidence of the impact of the project on the local community. In fact, the project research is rife with questions that have gone unasked.
The project is an example of the perhaps necessary tendency in development to fetishize success. Among community economic development practitioners, the Mondragon worker-owned co-operatives in the Basque region of Spain are often presented as examples of successful, democratically-run businesses. In the early 1990s I worked with welfare recipients in southern Ontario, attempting to help them start worker-owned co-operative businesses. The reality for most would-be entrepreneurs is that establishing a worker co-op presents some very significant challenges not found in sole proprietorship or partnership business models. Mondragon is a "success story," but for most people it is not a model that can be emulated.
While most development challenges are complex problems, and solving them is very difficult, there is a clear tendency in development research to over-emphasize success, even when projects fail. This was evident after the funding of telecentre projects of the 1990s. It is well known that most of these projects were not sustainable, and in fact that sustainability had not been factored into project and research plans. Yet there is little indication in published reports from this period of this important learned lesson. In fact, project literature might tend to report the few successful telecentre projects, and ignore projects that failed. International development agencies are under increasing pressure to create positive models for development and to indicate the value of Northern investment in development efforts.
I spent several months looking at international development ICT theory and practise, and with each paper became increasingly concerned about what I can only describe as the validity of the development research project. Several questions arose which I am still unable to answer:
- What constitutes "assistance" or "help" in a development context? Common sense notions of assistance become increasingly problematic as the complexity of a problem increases. Saving a drowning child is very likely to be helpful, but how does one increase the economic potential of poor African farmers
- Related to the above, what constitutes "success" in development? During the Green Revolution the capacity of Southern agriculture was increased substantially, but at the cost of very significant social and environmental disruption. What is a proper balance between improvements in some areas and disruptions in others? Can large-scale development take place without significant social costs?
- Given the complexity of most development problems, should we even expect "success" in the majority of projects? Development practitioners are understandably uncomfortable with reporting failure, but it is from these failures that the most important development lessons can be learned.
- Is advocacy research, or the funding of development in the guise of research, result in valid results in terms of either research or development?
- What sort of conclusions can we really draw about the transformative power of information technologies, or mass media? We know that some change has come about, but can we eevr be certain what?
