Can product curation overcome challenges in MSME market expansion? Opportunities for impact evaluation

Access to digital financial services (DFS) can help drive economic development by providing new, accessible digital tools and opportunities for the unbanked and newly banked. To increase DFS adoption, the Inclusive Financial Innovation Initiative (IFII) at J-PAL Southeast Asia (SEA) has supported exploratory and pilot studies led by researchers from the J-PAL network to generate evidence in high-priority areas. Through these studies, our goal is to maximize the potential benefits of DFS for economic development. 

Such exploratory and pilot studies have aimed to explore issues in DFS adoption, such as: 1) harnessing competition to improve the quality of branchless banking services, 2) addressing barriers to the adoption of digital lending among women microcredit clients, and 3) micro, small, and medium enterprises’ (MSMEs’) low visibility on e-commerce platforms. 

On 26 January 2023, IFII hosted a learning collaborative to share insights from exploratory and pilot studies to inform policy and decision-making in the digital financial inclusion space. Speakers and participants from government agencies, nonprofit organizations, and the private sector had the opportunity to discuss project findings and exchange insights on ways to increase the sustained adoption of DFS in Indonesia.

Below are the key takeaways from the learning collaborative:

1. Understanding how competition affects branchless banking agents' behavior is important for informing agent sales strategy 

Branchless banking agents are an important component of accelerating financial inclusion in Indonesia. Since 2014, agents have expanded to serve people without access to banks, creating a more competitive services landscape. Competition between agents may generate benefits or create obstacles. The potential benefits are that consumers will have more choices between agents, receive better customer service, and experience less misconduct. However, competition may also make business unsustainable for the agents due to the increased competitive pressure. 

Understanding how important these costs and benefits are in practice is essential for shaping banks’ agent sales strategy. To answer this, J-PAL affiliated professors Erika Deserranno and Firman Witoelar Kartaadipoetra, together with Gianmarco Leon-Ciliotta, Martin Kanz, and Daniel Gottlieb, are continuing their long-standing collaboration with a state-owned bank in Indonesia to conduct a study on how branchless banking agents respond to competition.

To understand the conditions of the branchless banking agents’ market, researchers began the study by conducting exploratory surveys of around 800 agents and 1,500 customers. Through these activities, researchers found that the majority of agents do not consider customers’ perception of them to be important, meanwhile customers value trustworthy agents. Hence, in anticipation of increased competition driven by market expansion, researchers now are designing interventions to study how informing different levels of competition affects agents' behaviors and services toward customers.

2. Increasing confidence and combating misperceived risks are key to increasing digital lending adoption among female microentrepreneurs

In Indonesia, 52.9 percent of microenterprises are run by women. DFS can potentially help businesses improve their access to financing, suppliers, and buyers. Despite the potential benefits, many women-owned businesses are still excluded from DFS. With many women being microcredit borrowers in Indonesia, there are opportunities to deepen their digital engagement and DFS adoption through loan digitization. Through a collaboration with an Indonesian microcredit institution and an e-wallet platform, J-PAL affiliate Arya Gaduh, with Arief Wibisono Lubis, conducted an exploratory study to assess such opportunities. 

The exploratory study was conducted to understand women’s barriers to digital technology and DFS in order to inform potential interventions designed to enhance their adoption of digital lending. Key findings from the exploratory surveys highlighted that women have a strong motivation to utilize digital technology for their businesses. However, surveyed microentrepreneurs often lack confidence and face challenges in navigating more complex apps like e-commerce and e-wallets. These women were also found to have concerns regarding the security risks of digital technology. These insights suggest that training and mentorship may accelerate women’s use of digital lending technology. At the same time, identifying other compelling use cases of DFS among these groups of women may be important to further build their motivation and digital engagement.   

Participants at the forum also discussed ways to better sustain DFS use, as microentrepreneurs frequently cash out their digital money because the transactions they need to make for their enterprises (with suppliers and consumers) are cash-based. One possible solution could be to introduce micro-entrepreneurs to the broader digital ecosystem, such as e-commerce vendors, to help lower their cost of production.

3. Signaling product quality may support MSMEs’ e-commerce success

E-commerce offers MSMEs the opportunity to access new markets and financial tools and services. However, in order for this opportunity to be realized, firms must thrive on the platform—and many struggle to do so. Thus, aside from encouraging MSMEs to enter e-commerce platforms, it is essential to ensure that they remain competitive once they are in the online marketplace. 

In 2022, as many as 17.5 million MSMEs in Indonesia entered e-commerce platforms. Yet, many MSMEs have failed to generate their first sales after joining due to inadequate promotional abilities, poor product quality, and/or a lack of digital literacy skills. More broadly, small firms are often difficult to locate on platforms, and consumers may be wary of purchasing from a small supplier with a limited track record. These “search frictions” limit many promising MSMEs’ ability to thrive on platforms.

To further explore ways to enhance MSMEs’ growth in e-commerce, J-PAL affiliates Jie Bai and Jing Cai, together with Daniel Xu, Chaikal Nuryakin, and Prani Sastiono, conducted a pilot study in collaboration with a large e-commerce platform in Indonesia. The pilot aimed to understand the importance of reducing search frictions through advertisements and coupons. The idea behind this approach is to increase the visibility of MSMEs and drive demand for their products. However, preliminary findings indicated that advertising MSMEs’ products on microsites or through banners, as well as offering vouchers for price and shipping discounts, did not lead to more sales. Other factors, such as MSMEs having less competitive value propositions due to their inability to signal good product quality, may explain the minimal impact. 

These initial findings suggest that more measures to signal product quality to customers may be needed to make MSMEs more competitive. There are different ways to indicate product quality, such as by applying for product certifications or improving the quality of product marketing to boost attractiveness and drive demand. Training for MSMEs can also support them in applying for certifications and in increasing the quality of their product marketing.

Finding innovative solutions to maximize DFS impact 

Overall, the learning collaborative forum gave important insights to create innovative solutions in pushing the frontier of digital financial inclusion. The findings presented from exploratory and pilot studies helped generate rich learnings that are useful for policymaking and future program designs. Going forward, J-PAL SEA’s IFII will continue to support stakeholders in finding innovative solutions to advance digital financial inclusion through rigorous evaluation.

Since officially launched in 2020, J-PAL SEA’s IFII has been working towards its long-term objective of ensuring that digital financial services drive economic development while lifting up low-income households that often lack access to formal financial services. To learn more about IFII, reach out to [email protected].

In Indonesia, many micro, small and medium enterprises (MSMEs) face challenges in accessing credit to grow their business, with only 12.7 percent of MSMEs borrowing from banks and other sources in 2021. These constraints are partially due to MSMEs' lack of collateral or historical credit information, leading lenders to view them as high risk or charge high interest rates. 

In response, firms using innovative credit-scoring (ICS) have emerged to help banks and peer-to-peer lenders predict potential borrowers’ likelihood to repay a loan. In contrast to traditional credit scoring models that use credit history, ICS uses artificial intelligence (AI), specifically machine learning algorithms, to evaluate data unrelated to a borrower’s credit history—what we refer to here as “alternative data”—to make predictions. These data could be digital transaction data, mobile phone and social media usage, or bills and tax payment history. 

Machine learning algorithms analyze large sets of the alternative data to identify patterns that can be useful in making credit risk assessments, which traditional models may overlook. For example, research shows that the presence of a financial app on an individual’s phone can be an indicator of financial sophistication and therefore, a useful variable for predicting credit worthiness. The accuracy of these algorithms increases over time as new historical data are added to the model. 

Adoption of ICS is a promising approach to increase credit access, especially for those who are unbanked or lack collateral. The Government of Indonesia has announced that they are exploring implementing a credit scoring system that uses alternative data to help more MSMEs gain access to lending. However, several questions remain: How can ICS effectively assist governments and lenders in increasing credit access? How accurate is the information generated by ICS, considering the diverse demographic of Indonesia? Are there risks associated with using ICS? 

These were some of the key questions explored during the recent “Learning Collaborative: Innovation for MSME Development” event hosted by J-PAL Southeast Asia and the Directorate of MSME and Cooperative Development of Indonesia’s Ministry of National Development Planning (Bappenas) on 11 December 2023. During the session, representatives from government, the private sector, academia, and non-governmental organizations discussed the opportunities and challenges of using alternative data and AI in credit scoring by drawing from existing research and their experience. 

Opportunities and challenges of using AI and alternative data 

Alternative data, such as mobile and social footprints, may enhance traditional credit scoring models and improve financial inclusion. A study in India by Agarwal et al. (2019) found that machine learning models using mobile and social footprint variables (e.g. types of apps installed, presence of social apps) can predict loan defaults more effectively than models that only use credit scores as they capture unique insights into credit risk that are not easily captured by earnings, education, or credit scores. The study also suggested that using alternative data can improve credit access for individuals without a credit score who would have been rejected by the traditional credit screening model. 

While alternative data may be used to predict loan default, participants at the Learning Collaborative event raised the challenge of data integration and transfer that may affect the performance of ICS, which highly depends on the quality and quantity of data. Currently, acquiring high-quality alternative data remains a challenge in Indonesia, due to hesitancy from data owners to share information, a lack of understanding of the methods to share data safely (i.e. protecting personal information), and the absence of a standardized data collection protocol among firms. 

At the moment, the government of Indonesia is creating an integrated MSME database that could facilitate data standardization and sharing and has developed the Personal Data Protection Law and AI code of ethics to govern safe AI use. Moving forward, participants also highlighted the importance of strengthening the regulatory landscape to enhance data management practices.  Aligned with these concerns, the Financial Services Authority of Indonesia (OJK) recently issued OJK Regulation No.3/2024 to support financial technology innovations, whilst ensuring consumer and data protection. 

Machine learning algorithms may be able to work alongside human agents and improve credit assessments. A study by Bryan et al. (2023) found that loan officers may not accurately assess potential clients, leading to suboptimal lending choices. In the study, loan officers incorrectly perceived clients predicted by machine learning as top performers to be more likely to default than those predicted as low performers. These studies suggest that loan officers may benefit from better identification methods, such as machine learning to make better credit decisions. 

However, while machine learning algorithms may be used to measure creditworthiness, there are biased risks associated with it. A study in Pakistan by Kisat (2021) revealed that including gender in machine learning’s dataset led to the algorithm selecting fewer women and thus, suggested that not revealing gender information may boost more equitable credit allocation. Similarly, a report by the Center of Indonesian Policy (CIPS) also cautions against biased risks. 

In Indonesia, biases may arise due to women’s underrepresentation in training datasets, as men are more likely to own smartphones and have household bills associated with their names. Therefore, it emphasizes that ICS firms must ensure that gender does not affect credit scores. 

Research questions to move forward 

Despite the increasing interest in ICS methods, there is limited research on AI’s application in MSME credit evaluations, especially in Indonesia. Further research can help guide ICS firms in developing algorithms that can boost financial access for MSMEs and minimize biases. It can also assist governments in creating policies and regulations that can facilitate credit allocations safely, efficiently and equitably. 

Areas for further research may include: 

Mapping out alternative data for credit risk assessment 

• Which types of alternative data are most effective for assessing the creditworthiness of entrepreneurs with varying socioeconomic backgrounds?  

Evaluating the implementation, performance, and cost-effectiveness of ICS 

• How can financial service providers effectively integrate the assessment of ICS with those made by loan officers? 
• How does the predictive accuracy of machine learning compare to that of loan officers? 
• Is using machine learning more cost-effective than employing loan officers? How can ICS and human judgments be effectively combined?
• What strategies can increase MSME owners’ awareness about ICS to help them dispute unfavorable credit decisions and take corrective actions? 

Understanding the discriminatory risks in credit scoring

• How do machine learning algorithms and loan officers respond differently to diverse demographic profiles? 
• How can the discriminatory risks of machine learning be mitigated? 

J-PAL SEA’s Innovation for MSME Development Initiative (IMDI) funds policy-relevant research and promotes knowledge-sharing among practitioners and policymakers who champion the use of rigorous evidence in the MSME sector. The IMDI team is open to discussing existing evidence, policy lessons and potential research questions and brainstorming ideas to advance MSME growth. To get involved, reach out to Rizka Diandra Firdaus (Policy and Communications Manager).

This is the fourth post of J-PAL’s 20 for 20: Partner Voices blog series, where we showcase stories of results and impact with our partners to celebrate our twentieth anniversary. Read on to learn more about Mohammad Zeqi Yasin’s, head of research at the Research Institute of Socioeconomic Development, experience collaborating with J-PAL Southeast Asia on a large-scale exploratory survey of vocational school teachers.

Since 2020, I have been working as the head of research at the Research Institute of Socioeconomic Development (RISED), a think tank based in Indonesia. In this role, I, alongside two other project officers, had the opportunity to collaborate with J-PAL Southeast Asia (SEA) on a large-scale exploratory survey of vocational school teachers, which aimed to investigate the impact of innovative vocational teacher training on improving teacher's vocational sectoral knowledge and teaching practice. 

In the exploratory stage, we conducted more than 1,600 phone survey interviews with teachers. Our collaboration with J-PAL SEA was a pivotal experience for our team—it presented us with the opportunity to gain valuable insights into ethical and effective research practices, and gain new skills in data collection in field and primary research. 

Learning about the best practices for ethical research 

To kick off our collaboration, J-PAL SEA conducted a training with our research staff and enumerators focused on the essential ethical research practices employed by J-PAL, known as "minimum must-do's" (MMDs). The training covered a wide range of standard protocols in administering phone survey— including: 

• Research principles around human subjects, child protection, and gender mainstreaming; 
• Phone survey techniques and other survey instruments; 
• Simulation of data storing and cleaning, and data confidentiality training (e.g. data encryption and data de identification). 

The training prompted our team to apply MMDs in research practices that we had not previously used before. For example, it underscored the importance of ensuring data security, which involves using standardized tools such as laptops equipped with reliable anti-malware and antivirus software, as well as implementing data encryption measures to avoid data misuse and maintain confidentiality of sensitive data. The training also provided us with new insight on how to incorporate enumerators’ feedback to enhance the survey instruments. In addition, this training emphasized the significance of ethical considerations during field surveys when working with human subjects, a dimension often overlooked by research organizations. In particular, from the beginning of the training J-PAL SEA emphasized  the importance of being aware of the sensitivities when interacting with certain groups, such as children and women. 

Applying the research practices in the field

The survey collection process spanned over three months. At first, it was challenging to keep up with the great amount of new information, standard operational procedures, and terminologies. However, we were able to apply the MMDs during the phone surveys due to our adamance and the guidance from J-PAL SEA. The J-PAL SEA research team actively collaborated with us to provide guidance on issues that arose during the field survey and were also open to receiving constructive feedback from the field officers. 

One of the challenges that arose during the online survey period was some enumerators were not able to meet their target of 200 respondents. This issue came up due to inactive phone numbers in the list of possible respondents and difficulties following up with respondents even after they secured an interview appointment. 

To mitigate this, the J-PAL SEA and RISED teams collaboratively held a debrief session, which created a useful platform to exchange best practices on how some enumerators managed to complete more interviews. Additionally, enumerators who already reached the targeted 200 interviews were given the option to survey another list of respondents. This strategy helped the team to collectively achieve a total of 1,600 completed interviews.

Opportunity: Fostering the development of domestic research institutions  

Overall, this experience provided us with insights into the best practices for conducting ethical research and encouraged us to employ the MMDs as standard procedures in our future projects. Looking ahead, as an international research institution with extensive experience in data collection, J-PAL SEA can foster the development of domestic research institutions by sharing its knowledge and encouraging research institutions to incorporate rigorous data collection standards into their practices.