Diverse AI Teams and Responsible AI: Why Representation Is Only One Safeguard
November 1, 2023 · Human Resources, Recruitment · 9 min read
Diverse teams can improve the range of perspectives involved in building artificial intelligence, but representation alone does not guarantee that an AI system will be fair, accurate or ethical.
Responsible AI requires both inclusive participation and risk-management discipline: clear objectives, appropriate data, testing across relevant groups, transparency, accessibility, human oversight, impact assessment and monitoring after deployment.
This is especially important when AI is used in employment, where automated systems can influence who sees a vacancy, who is shortlisted, how candidates are assessed, how workers are scheduled and how performance is evaluated.
Why diversity still matters in AI development
AI systems are socio-technical systems. They are shaped not only by code, but by decisions about:
- which problem is worth solving;
- how the objective is defined;
- which data is collected;
- which outcomes count as success;
- which harms are considered;
- who is affected;
- which trade-offs are accepted.
A team with a narrow range of experience may be more likely to miss an important user group, accessibility issue, cultural assumption or harmful edge case.
UNESCO’s Recommendation on the Ethics of Artificial Intelligence places diversity, inclusion, fairness, transparency and human oversight among the core principles of responsible AI.
Source: UNESCO, Recommendation on the Ethics of Artificial Intelligence
Diverse teams are a safeguard, not a guarantee
It is possible to have a demographically diverse team and still build a harmful system. It is also possible for a relatively homogeneous team to identify some risks through strong governance and external challenge.
The stronger approach is to combine representation with:
- multidisciplinary expertise;
- stakeholder participation;
- documented risk assessment;
- representative testing;
- clear accountability;
- independent challenge where appropriate.
Start with the objective the AI system is optimising
Before looking at the model, ask what the system has been told to optimise.
The International Labour Organization’s 2025 working paper on AI in human-resource management warns that AI systems can encode poorly defined objectives, rely on biased or incomplete data and operate through opaque processes.
Source: ILO, AI in Human Resource Management: The Limits of Empiricism, 2025
A recruitment system designed to predict “culture fit”, “growth mindset” or “future potential” may appear objective while operationalising a vague human concept through weak proxies.
Ask:
- What exactly is the system predicting?
- Why is that outcome relevant to the job?
- What evidence supports the proxy being used?
- Could the objective disadvantage a group for reasons unrelated to performance?
Data quality is an ethical issue
AI systems learn patterns from data. If historical data reflects unequal access, biased decisions or underrepresentation, the system can reproduce those patterns.
Data problems can include:
- missing groups;
- poorly labelled outcomes;
- historical bias;
- proxies for protected characteristics;
- data collected for a different purpose;
- different quality across populations;
- language or accessibility gaps.
The question is not only whether the dataset is large. It is whether the data is appropriate for the decision the system is being asked to support.
Employment AI deserves especially careful scrutiny
Hiring and workforce-management systems affect access to jobs, income and career opportunity.
The ILO’s 2025 analysis of AI in HR highlights risks in recruitment, compensation, scheduling and performance management, particularly when systems are built on reductive objectives or opaque data-driven assumptions.
The ILO also notes that AI can potentially support fairer and more transparent HR processes when it is designed and governed responsibly.
Inclusive recruitment teams should not rely on intuition alone
When recruiting AI professionals, organisations can reduce arbitrary decision-making by using:
- job-related selection criteria;
- structured interviews;
- consistent scoring rubrics;
- skills-based assessments;
- multiple reviewers for important hiring decisions;
- accessible recruitment processes;
- monitoring of progression through the recruitment funnel.
Diversity goals should not replace job-related assessment. The objective is to widen opportunity and reduce unnecessary barriers while maintaining a clear standard for the work.
Remove unnecessary barriers from AI job descriptions
AI teams need a range of roles, not only machine-learning researchers.
Depending on the system, relevant expertise can include:
- software and data engineering;
- statistics;
- product management;
- cybersecurity;
- privacy;
- law and compliance;
- human factors;
- accessibility;
- domain expertise;
- ethics and social research;
- operations;
- user research.
Overly narrow degree, pedigree or experience requirements can shrink the talent pool without improving job performance.
Multidisciplinary teams can see different classes of risk
An engineer may identify model-performance issues. A domain expert may recognise that the target variable is meaningless in practice. An accessibility specialist may identify a barrier for disabled users. A privacy professional may identify unnecessary data collection. A frontline employee may show that the workflow does not match real work.
This is why NIST describes AI risk management as a multidisciplinary and socio-technical activity rather than a purely technical exercise.
Source: NIST AI Risk Management Framework
Include affected people, not only internal experts
Internal diversity is useful, but the people building the system are not always the people most affected by it.
Depending on the use case, stakeholder participation can include:
- employees;
- job applicants;
- customers;
- worker representatives;
- people with disabilities;
- community organisations;
- subject-matter experts;
- people from affected language or cultural groups.
Participation is most useful when stakeholders can influence design choices rather than being consulted after the important decisions have already been made.
Test performance across relevant groups
An overall accuracy score can hide important differences.
Responsible testing may need to examine:
- false-positive and false-negative rates;
- performance by relevant demographic group;
- language performance;
- accessibility;
- edge cases;
- different devices or environments;
- different levels of data quality.
Which groups are appropriate to test depends on the use case, population and law.
Accessibility should be designed in from the start
AI-enabled recruitment can create barriers for candidates with disabilities when systems assume a narrow range of speech, facial movement, interaction style, device use or response format.
The ILO has highlighted that AI can both improve accessibility and create exclusion depending on how systems are designed and used.
Source: ILO, AI’s double-edged sword and disability employment
Organisations should provide accessible alternatives and a process for reasonable accommodation appropriate to the jurisdiction.
Do not assume automation removes human bias
An automated system can reproduce human bias through historical data, labels, proxies or design choices. It can also create new forms of bias through optimisation and scale.
Human review is still important, but “human in the loop” is not enough if the reviewer simply accepts the system’s output.
Reviewers need:
- authority to override;
- information about limitations;
- time to examine cases;
- a process for escalation;
- accountability for the final decision.
Human oversight should be meaningful
UNESCO’s AI ethics framework emphasises that AI should not displace ultimate human responsibility and accountability.
Meaningful oversight asks:
- Who owns the decision?
- Can a person explain why the system was used?
- Can the affected person challenge an outcome?
- Can the system be paused if harms emerge?
- Who monitors performance after deployment?
Use impact assessment before high-risk deployment
UNESCO has developed an Ethical Impact Assessment tool to help organisations examine alignment with its AI ethics principles.
Source: UNESCO, Ethical Impact Assessment
Before deploying a consequential system, assess:
- intended purpose;
- affected populations;
- possible benefits;
- foreseeable harms;
- data sources;
- performance limitations;
- human oversight;
- redress;
- monitoring;
- conditions for stopping use.
Monitor the system after deployment
A system that performs acceptably in testing can behave differently when:
- the applicant pool changes;
- job requirements change;
- language changes;
- data drifts;
- users adapt their behaviour;
- the vendor changes the model.
Responsible AI requires ongoing monitoring, not a one-time fairness test.
Vendor tools still require employer accountability
Buying a recruitment or HR AI product does not remove the organisation’s responsibility to understand what it is doing.
Ask vendors:
- What objective does the model optimise?
- Which data was used?
- How has the system been tested?
- Which populations were included?
- What accessibility testing was performed?
- What information can be given to affected people?
- How are model changes communicated?
- Can customers audit outcomes?
- How is data retained and protected?
Diversity metrics should cover the employee lifecycle
If an organisation wants a more representative AI workforce, recruitment is only the first step.
Track, where lawful and appropriate:
- applicants;
- shortlists;
- offers;
- acceptance;
- retention;
- promotion;
- pay;
- access to important assignments;
- leadership representation.
A company can hire a diverse entry-level cohort and still lose diversity through promotion or retention patterns.
Create a team culture where risk can be raised
Representation is less useful if people do not feel safe challenging a product decision.
Leaders should make it legitimate to ask:
- Who could this harm?
- Which population are we missing?
- What assumption are we making?
- What happens if the model is wrong?
- Should this decision be automated at all?
A practical responsible-AI staffing model
For a consequential AI project, consider whether the team has access to:
- technical AI expertise;
- domain expertise;
- data governance;
- security;
- privacy;
- legal/compliance expertise;
- accessibility;
- human factors or user research;
- affected-user or worker perspectives;
- senior accountability for deployment.
The exact mix depends on the system and risk.
Responsible AI is a process, not a diversity statement
Diversity can make AI teams better able to see risks and serve a wider range of users. But ethical outcomes require an operating system around that diversity: governance, testing, accountability and real stakeholder participation.
The strongest question is therefore not “Is our AI team diverse?” It is “Do we have the perspectives, processes and controls needed to identify and manage the risks this system creates?”
MATSH and responsible workplace technology
MATSH provides professional-development content across leadership, HR, diversity, technology and workplace capability. Organisations adopting AI in HR should combine technical expertise with strong people-management, fairness and governance capability.
Frequently asked questions
Does a diverse AI team automatically produce unbiased AI?
No. Diversity can broaden perspective, but fairness also depends on objectives, data, testing, governance, oversight and monitoring.
Why can AI hiring tools be biased?
Bias can enter through historical data, labels, proxies, poorly defined objectives, non-representative testing or the way the system is deployed.
Should employers use AI in hiring?
AI can support recruitment, but employers should assess whether the tool is job-related, accurate, accessible, transparent enough for the use case and governed appropriately under applicable law.
What is the role of human oversight?
Humans should retain meaningful responsibility for consequential decisions, understand system limitations, be able to override outputs and provide a route for challenge or redress.
What should an AI ethics team include?
There is no universal structure. Consequential projects often need technical, domain, privacy, security, legal, accessibility, human-factors and affected-stakeholder perspectives.
Related MATSH resources
- Digital Skills for Professionals: What You Need in 2026 and Beyond
- AI and Technology in Professional Training
- Cybersecurity Upskilling in African Banking
Sources
Need In-House Training?
We run all our courses as private programmes for organisations across the GCC and Africa.
Request In-House →