Ethical Dimensions of Nonacademic Research in the Development Sector: A Perspective from Jordan

Several years ago, I was commissioned to lead a study related to women’s community leadership roles. As part of the study, I was responsible for conducting a focus group in the remote Azraq refugee camp in Jordan where the lives of thousands of Syrian refugees have been put on hold for years. I had previously conducted focus groups and interviews in both Azraq and Zaatari refugee camps. Although I did not know it when I got the assignment, this focus group was planned for Village 5, a secured prisonlike area that hosts nearly ten thousand Syrians. While refugees in other parts of the camp can be granted work or leave permits that allow them to enter the rest of the country even for a week’s break, Syr­ians in Village 5 for reasons of national security and safety are not allowed to leave.

At the camp, I ended up in a small room with eleven disheartened middle-aged men, all beneficiaries of an international nongovernmental organization (INGO) that operates there. After explaining the purpose of the study and obtaining their consent to participate, I proceeded to ask questions from the approved discussion guide. It immediately became apparent that the research topic I was there for was entirely irrelevant to this group of incarcerated men living in heart-wrenching conditions. Seething with anger, the men did not want to speak about anything but the immediate needs of their children and their sense of helplessness and hopelessness. They spoke of the majority of village kids who were without shoes, and the limited access to water, employment, and health services. A heartbroken father said he was contacted about an MRI he had requested for his sick son, but the call had come after his son had already died. I was not ready for this and perhaps that was partly my fault. I had not been warned by anyone and did not anticipate this moment. No one from the INGO that provides them with vital services and support had explained to them what the study they had been gathered for was about. The men had no choice but to show up there. Their consent and autonomy were clearly compromised. They did not have a meaningful choice to participate in this focus group. I pushed past my emotions, indignant at the research project that was oblivious to their plight. I explained again that I was not in a position to provide the assistance they were expecting. Seized with the strongest regret, I promised to relay their messages, asked very few questions afterwards, and let them vent for the remainder of the time. Enervated by this experience, I left defeated and heartbroken myself.

The stress and discomfort my presence had caused this group was sufficient to constitute emotional harm. It was not justifiable and could not be outweighed by any benefits to the research subjects. Had I insisted on asking my irrelevant questions about esoteric leadership constructs in this survival setting, I would have done them more harm. My presence there was cruel. It had shown a callous insensitivity to their vulnerabilities and needs.

Beyond the personal moral intuition that undergirds the ethical conduct of all research, a set of universal principles is supposed to guide the practice of research and the behaviors of researchers to promote its integrity and ensure that studies are undertaken in ways that protect and enhance the interests of the participants.1 Various ethical frameworks and professional codes of ethics have prescribed a threshold of ethical behavior that the conduct of research should meet. These codes, however, do not address all ethical challenges that one may encounter when conducting research, and the connection between abstract ethical theories and practical issues is not always clear.2 As researchers, we often find it difficult to reconcile abstract standards with the messy context in which research is conducted.3 The inadequacy of codes may sometimes be resolved by reference to higher-level ethical principles, including beneficence (do good), nonmaleficence (do no harm), respect for persons, justice, and fidelity (keep promises and don’t lie).4 However, in the absence of local interpretation, endorsement, and regulation, inchoate ethical considerations can be easily feigned. The application of universal ethical constructs may become a form of ethical window dressing that undermines research relevance, value, and impact and leaves a litany of consequential ethical questions unanswered.

Scholars sometimes distinguish between two levels of ethical principles: while internal principles address the ethics of studies for the participants, external principles cover the ethics and safety of the study for the community.5 The latter requires a broader community perspective and is concerned with consequences of the research project and the potential impact of the project on community members, even those who did not participate in it directly.6 Absent that, research ethics can be narrowly interpreted and applied, or restricted to a few checkbox actions related to the interaction with participants in the field. In a context like the refugee camps in Jordan, the notion of external ethicality is of particular relevance and importance. It requires a broader consideration of ethical challenges related to how the research is framed, justified, funded, and used within the local context where it is conducted.

After all, the ethical conduct of research is an integral part of its quality.7 Research quality is multidimensional and is generally assessed at the three stages of research practice: design, process, and impact.8 Various evaluation metrics are used to assess the quality of research practice, which range from traditional quantitative and bibliometrics-related criteria to qualitative and expert judgment. In the development sector, donors and INGOs define or influence research quality at all three stages. They also determine the integrity standards of any research conducted or commissioned.

A variety of international development and humanitarian actors operate in Jordan, a country battered by regional crises and weighed down by high unemployment, poverty, and millions of refugees. Because of its resources, the aid industry dominates the ecosystem of knowledge production with a pipeline of outputs that include different study types and methodologies. With variable quality, research is fragmented across several donor fiefdoms while limited coordination and regulation consume resources without sufficient justification. Most donors conduct their own needs assessments, baselines, and gender and stakeholder analyses, among other studies, often with repetitive results. This context is a testament to Jordan’s particular situation, in which international aid organizations linked to deep-pocketed donor governments enjoy considerable leverage and independent action. Questions of how allocations of limited resources are determined, how research priorities are set, how selected studies fit within the broader national development agenda, and how these studies contribute to the state of knowledge of sector challenges (and how to address them) are often left neglected. Without consideration of these issues, the ethicality of many of these studies is questionable.

Development-sector research reflects the same asymmetrical power dynamics of the broader development sector in Jordan. Donors decide which development programs to fund and then contract international or local NGOs to implement them. Research calls are usually associated with specific ongoing or upcoming development or humanitarian programs. With a largely top-down approach to agenda-setting, international donors determine research priorities, the scope of commissioned studies, and their tone on publication. The studies are either conducted directly by the INGO or subcontracted to local organizations, which can be local NGOs, for-profit development consulting firms, or university research centers. The research is most often intended to inform the design and implementation of development programmatic interventions or assess their performance.9

Current hierarchical forms of accountability prioritize funders at the expense of local development actors and assistance recipients. Research priority-setting processes are opaque to local stakeholders. Through their implementing INGOs, donors solicit proposals that address research questions related to their development programs. Donor agencies are not required to justify the need for the studies they commission. The broad program outlines, however, are approved by the Ministry of Planning and International Cooperation.

Donors are not required to respond to knowledge gaps, build on existing knowledge, or be meaningfully responsive to national priorities. While there are exceptions, country-knowledge is less valued and usually trivialized.10 The head of a consulting firm said he believed that local knowledge is “looked down upon when deciding research priorities.”11 Another local researcher described how local knowledge was dismissed in a recent needs assessment that was conducted in three communities in Jordan: “They wanted the assessment to focus on three communities but the Ministry of Social Development believed other marginalized communities should be prioritized. The assessment ended up going for the communities the INGO selected.”12

Considerable overlap and fragmentation characterize the development sector in Jordan with donors and INGOs commissioning studies that may not recognize or build on previous research on the same topics. “There is considerable competition between the INGOs, even those contracted as part of the same development program,” said a representative of a large national NGO. He noted that his organization is regularly asked for “fresh” studies related to youth and women’s empowerment even though “similar studies” were recently conducted by other organizations. Studies are sometimes a means to justify large program budgets, as two interviewees agreed. When you have a five-year multimillion-dollar program, the implementing organization needs to build in several assessments and studies to help justify the large budget. “There is no control; they work independently. The resources they spend on endless studies can be spent more meaningfully or just donated to communities,” said an owner of a development consulting firm.13

Various local researchers often complain that organizations in the sector are oblivious to the research other actors in the sector conduct. There are also no information sources about other research in progress. Donors often complete a study only to find out that another donor is about to complete a study with a similar focus. This contributes to research redundancy and the waste of needed resources. “I just received three requests for studies that are similar to something I’m already working on,” reported the head of a consulting firm.14 He further described the sector as “frivolous.” With so much aid money coming in, there is little imperative for efficiency.15

Donors play a key role in aid-recipient countries. They shape national policies by negotiating priorities and channeling and controlling aid.16 In effect, their funding criteria determine targeted beneficiaries and areas.17 Such dynamics tend to disrupt local NGOs’ missions and their ability to develop their research footprint organically. The development assistance ecosystem pushes local organizations to be service contractors, limiting their ability to develop their own action research agendas. As their primary source of financial support, foreign assistance that is linked to donor-led development programming cuts off opportunities for developing independent scholarship that addresses local challenges and provides policy advice. Local organizations receiving aid in the form of patronage lose their autonomy and carry on their funders’ agendas.18 A representative of a national NGO with a research unit said his organization does not have a research agenda. “We only respond to RFPs [requests for proposals] but have a wish list of research subjects we try to pitch to donors.”19 Another interviewee agreed, adding that the reluctance of international development organizations to cover national organizations’ operational costs stunts the ability of local organizations to conduct independent research. Relatedly, the insistence of international organizations on owning the research data that local organizations produce also limits the latter’s ability to reuse or repurpose the data for subsequent research.20

Topic choices are justified in requests for proposals that local organizations cannot deviate from. “You have to stick to the RFP; if you don’t you will not get the assignment,” reported a head of a development consulting firm.21 Responding to an RFP, a bidding local individual consultant, consulting firm, or NGO will be assessed on their overall approach, methodology, team composition/expertise, and proposed budget. The winning service provider will then be required to submit a detailed study design and methodology to the donor/INGO, including all data-collection tools, before the research process can start. Bidding organizations and especially consulting firms are first and foremost motivated by profit margins, which can encourage them to sacrifice quality.

When it comes to the research process, various issues stand out. First, investment in new research should always be preceded by systematic assessment of existing evidence revealing the state of knowledge on a particular topic.22 In Jordan, however, the failure to view research and development systematically has in turn produced a lack of a systematically collated evidence base. Even when studies include a literature review phase to overview current knowledge, the review can lack rigor and thoroughness and return shallow results. This in part can depend on the length of time allotted by the donor or INGO for the research assignment. Some allocate timeframes that are insufficient to collect and synthesize available research, and can push the consulting firm or local NGO to cut corners for cost efficiency. A one-week literature review will expectedly be much weaker than a longer review that covers the same topic.

Second, the qualification requirements for the research consultants vary across international organizations, whether the research is commissioned by international donors or INGOs. Study teams usually include combinations of a team leader, one or more subject experts, specialists, and data collectors. The cross-field approach is meant to reflect the nature of societal challenges that are seldom solved by individuals within single specialist fields.23 Scholars with doctoral degrees sometimes lead study teams, but terminal degrees are rarely required. There is an increasing tendency to recruit national consultants to lead research projects, but some donors still require team leaders to be “international,” even for studies whose focus could easily be addressed by local or regional experts. With organizations prescribing specific academic and professional qualifications, the additional requirement of being “international” serves little but to illustrate the intimate linkages between development, inequality, and white saviorism. Flying in Western researchers to stay for weeks or months to lead studies also pushes up their already exorbitant cost, including the price to translate qualitative research transcripts. By contrast, study teams often rely on “data collectors” who tend to be junior staff with lower pay rates. NGOs and consulting firms usually train these staff before unleashing them on the field to collect both qualitative and quantitative data. The quality of such training can vary and is usually discretionary to the local organizations undertaking these studies.

For a recent program-evaluation study, the donor rejected a local expert proposed to lead the assignment only to later accept him after others pushed back, citing his dual citizenship. Two other recent calls for proposals by the same donor also required “international team leaders.” Commenting on her experience working as part of a local NGO on an assessment for an INGO, a Jordanian senior researcher with a doctoral degree said the INGO had insisted on bringing in an international gender expert “when we have so many excellent gender experts in Jordan.” She added that because the INGO used Google Translate to cut costs, the quality of the transcripts was so bad that she asked for the removal of her NGO’s name from the final research report.24

Third, studies are usually submitted to the INGO and/or the donor that commissioned the research. The studies are reviewed and commented on by the client’s staff, usually technocrats with varying degrees of capacity and local knowledge. Depending on the donor or INGO, the study could shuttle back and forth between its offices and those of the local NGO or consulting firm conducting the research with different rounds of feedback and revisions.

When foreign donors provide the funding and determine the research scope, our “permission to narrate” as local researchers can be circumscribed. For example, donors may ask for certain recommendations to be toned down, especially if they are directed at the government. Sometimes entire research studies are dropped if they are deemed “too sensitive.”25 Professional researchers can censor themselves or engage in tactical positioning to ensure their studies will survive donor scrutiny, and that of government, when required. As one NGO representative recounted:

A donor asked us to conduct three studies. I was asked to change some of the findings in one of them. We later discovered that the donor was working on the sector that is the focus of the study they asked us to change the findings of. They did not like our findings because they were critical of their work. I refused to change them. They can review the methodology or question it if they want but they cannot ask me to change the findings. So, they told us to pull their name from the study and publish it on our own.26

Research fatigue is a serious issue affecting data validity and reliability as well as people’s trust in research. Scholars have documented various trends that influence research in fragile and over-researched settings, with study participants seeing researchers not as independent actors but as part of a larger collective that includes aid workers or journalists.27 Jordan experiences a similar dynamic. Both refugee camps and host communities remain largely accessible to international organizations that regularly tap into their direct beneficiaries for data and validation of their programs’ logic frameworks. “Communities, especially where refugees reside, have become open labs,” reported an NGO representative.28

The quality of data is impacted by various types of biases that result from over researching, from the power and knowledge disparities between international and local development organizations, and between development sector researchers and research subjects who often view the research through the lens of the incoming aid. In such a context, standard consent forms cannot secure autonomous consent, especially for participants who have been socialized to expect development support from the research. These dynamics have considerable ethical considerations that are seldom highlighted or addressed, affecting both research quality and impact.

Sarah E. Parkinson describes mechanisms that shape interactions between researchers and research subjects in refugee camp settings, versions of which our consulting team has experienced regularly when collecting data across the development sector in Jordan.29 Regurgitation, for example, occurs when the participation of research subjects is shaped by patterns learned from previous interactions with researchers. Communities in Jordan, especially in areas where Syrian refugees concentrate, are usually targeted by multiple international assistance programs with their own research studies. Representatives of consulting firms said that regurgitation occurs as a result of research fatigue that Jordanians and Syrian refugees continue to experience as siloed donors roll out studies with limited coordination. The transportation stipend consulting firms or NGOs sometimes offer to incentivize research subjects to participate can also sway results through regurgitation.30

Similarly, Parkinson describes two other mechanisms she observed in refugee settings. The first is redirection: when participants try to leverage the research interaction to solicit favors or redirect the conversation to another topic they find more relevant to their daily lives or needs. Reluctant participation, the second type, refers to the exhaustion experienced in the development sector when individuals feel obliged to participate in research either to gain favors from development organizations or because they are current beneficiaries of development assistance.31 I have observed the operation of both mechanisms in nonrefugee settings across Jordan.

Over the last decade, “systems change” has gained traction as a more effective means to tackle development challenges. A systems change approach attempts to address the root causes of social problems, including the policies, attitudes, and practices that may be enabling them. The term considers these challenges as symptoms of deeper problems embedded in the ecosystems in which they operate and remain entrenched.32 But approaching complex development programs with such a lens requires a thorough understanding of the structures and mechanisms within the systems in which the challenges are situated, and their relationships and power dynamics.33 This in turn calls for an analysis of each system’s components and how they interact, with the aim of addressing bottlenecks and creating long-term value rather than short-term solutions.

Instead, many research studies in the development sector mirror development programs in their shortsightedness. Programs develop a logic of their own that shapes the allocation of resources and the activities that will be part of the program.34 They tend to focus on the limited aspect of reality they feel they can do something about.35 In such a context, commissioned studies tend to concentrate on sector-specific technical issues. Those are usually proximate causes of development challenges that avoid delving into messy governance contexts.36 Countless “capacity needs assessments,” for example, may tangentially touch upon systemic issues but more often ignore them outright. Resulting studies are meant to be less political and more “technical.” By being hyperfocused on the immediate universes of the programs, related studies feed simplistically linear development interventions that are devoid of contextual realities necessary to address bottlenecks at different system junctures. For example, capacity assessments often recommend the need for additional training of local actors. But these actors may include individuals and institutions that have limited mandates, agency, or tenure. During the decentralization reform in Jordan between 2015 and 2019, governorate and local council members across the country were trained again and again by different donors when the councils at both levels had experienced very limited authority. Local councils have since been canceled by law, and governorate councils still suffer the same limited mandate and remain starved for resources.

Research is meant to be used to “improve the condition of individuals, organizations, and society.”37 The improvement imperative is both difficult to define and capture. However, with development research closely linked to development programming, the question of its impact becomes that of the impact of development assistance itself. And while that issue is outside the scope of this essay, one can still focus on two metrics that can unpack the impact of development sector research. The first concerns the extent to which research is addressing a local learning agenda, and how relevant it is for local policymaking; and the second is related to knowledge-translation efforts and their successes.

The context in which policymaking is conducted in the region is characterized by a lack of transparency.38 In Jordan, the disconnect between the research community and that of policymaking is due to various factors that affect the ability of the two sectors to work together. These include a low demand for research by policymakers, the different sectoral cultures and data generation and use routines, the language barrier, and time limitations. Specifically, the weak demand for accountability shapes local organizational practice in Jordan and dampens the need for quality data and research use. Limited capacity can also dilute the value of research and disincentivize its use. On the flip side, a weak perception of the value of research will also discourage policymakers from building capacity for research.39 Recently, more international donors are trying to engage local actors in development program design, including the development of their research/learning agendas, but these one-offs are often limited in scope and characterized by desultory implementation.40

Development sector aid agencies and their contractors often publish their studies and disseminate them broadly. But this dissemination is passive, with limited impact on policymaking. For example, USAID established the Jordan Development Knowledge Management Portal, on which it uploads most of the studies it commissions, and has encouraged other donors and local actors to do the same. But USAID does not track traffic or activities or encourage engagement with content. Moreover, the studies are published solely in English, limiting their accessibility and uptake by local researchers, decision-makers, and organizations.

Beyond this, the research cycle for the local researcher (and research subjects) is disrupted by the completion of the study. Local researchers are rarely involved the day after they submit the studies to the donors or INGOs. This hampers their ability to engage as knowledge brokers and promote interaction with end users, including policymakers, and prevents them from sharing research results. “I don’t know what happened to the study or how they used it,” said a senior researcher.41

Increasingly the state of nonacademic development research in Jordan has become ethically indefensible. “The majority of development work is not well-researched. There is a lot of plagiarism because we don’t feel the monitor monitors.”42 The absence of a national research system to regulate both academic and nonacademic research, and that can define local research priorities, assess the quality of research outputs, and determine the parameters of sufficient evidence within the local context, has contributed to this situation. Improving the quality of research in the development sector and its ethical conduct requires a redefinition of the role of donors and INGOs to one that is unequivocally and measurably supportive of a timed localization process that addresses inequality and white saviorism in international development.

The ethics of academic field research is usually assessed through the process of an internal review board (IRB). The practice of ethical reviews has attracted a fair share of criticism, but it remains instrumental in safeguarding human subjects and ensuring overall research quality.43 Even though ethical review may be an important measure of research excellence, the majority of studies in the development and humanitarian sectors in Jordan do not undergo such reviews or any local standardized regulatory process or mechanism for ensuring ethical research. There are currently no national requirements that the research has to satisfy to ensure quality. “Even when the studies are done, peer reviews are rare; we are a small pool of researchers here and we all know each other.”44

Development programming can benefit from collaborating with academic institutions, which can help oversee and/or coproduce research from design to implementation to the translation of results. Such partnerships should be built on mutual learning and efforts to strengthen local capacity. There are good models to watch out for. To support localization of knowledge production and Southern research partners, the International Development Research Center (IDRC) has recently launched a network of research chairs based in twelve universities across three continents. The research chairs focusing on forced displacement aim to strengthen the institutional base of local knowledge production, helping to create more context-relevant solutions for displaced individuals and their countries.45

In 2019, an IRB was successfully established with a USAID grant at the Princess Sumayya University for Technology (PSUT) in Amman. The board developed its own guidelines adapted from The Belmont Report and currently reviews both academic and nonacademic research.46 Still not well publicized and not mandatory for all research involving human subjects, donors opt to avoid the IRB process. If reviews are requested, research proposals are usually considered against the basic principles of informed consent, minimization of harm, and privacy.47 Though in its early stages, the PSUT IRB recognizes the importance of establishing uniform ethical standards for the research community in Jordan and is already providing basic research ethics training to researchers and data collectors. However, the lack of a locally defined, owned, and adopted definition of research excellence, in addition to the absence of a national research system that can systematize or strengthen notions of research accountability, means the process has a long way to go. For one, such a system would act as a screening mechanism for all aid-related research to ensure need, compliance with ethical standards, and harmonization of related aid projects.

In the spirit of aid localization and more-effective development programming, including the research practice that informs it, it is imperative that the fledging IRB process in Jordan is propped up and its purview expanded. Considering the ubiquity of need for development aid and the number of related studies commissioned each year by donors and INGOs, a well-resourced independent research quality review body that considers the ethical conduct of the research as part of its expanded mandate is necessary. This body can undertake a retrospective reflection on the societal need the research is meant to address, define the ethical imperatives particular to this context, and ensure the efficiency, rigor, and value of commissioned research, with attention to transparency, respect to stakeholder voice, and ownership of the research agenda and results.48

As this local review body continues to develop standards, procedures, and policies to promote the ethical conduct of research, harm avoidance would need to include not only physical endangerment but also harm to intangible social assets such as relationships and public trust in research itself.49 Considering available local and regional expertise and the capacity-strengthening programming that donors and INGOs have been pushing for decades, development studies should also be led by researchers based in the region of study, while aid organizations pursue an intersectional approach in their continued capacity-building efforts to address various layers of disparities that local researchers face.

The ethical review body would need to be able to draw upon a repository of all development sector studies in addition to scholarly published research. Under an academic umbrella, the ethical review body should also be able to track and align with national research priorities and define development research needs. Considering the scarcity of resources, priority-setting is important. A national body fulfilling the functions of a local regulator can help ensure that all research proposals are justified by systematic reviews revealing the state of knowledge on a specific topic before new research is commissioned. This will not only save resources and eliminate research redundancy but will also support a more thorough engagement with available data to inform future research and programming. It will help allocate resources for the syntheses of existing studies rather than the commissioning of new ones.

Dissemination alone does not ensure the utilization of knowledge. It must be combined with activities to build opportunities and motivation for local actors to use research results.50 An impact agenda should be determined locally, moving away from a linear conceptualization and instead focusing on an integrated development approach that emphasizes systemic change and measurable progress toward development goals and their targets.