Algorithmic Justice League: Gender Shades

How well do IBM, Microsoft, and Face++ AI services guess the gender of a face? The Gender Shades project evaluates the accuracy of AI powered gender classification products.

https://www.youtube.com/watch?v=TWWsW1w-BVo

This evaluation focuses on gender classification as a motivating example to show the need for increased transparency in the performance of any AI products and services that focused on human subjects. Bias in this context is defined as having practical differences in gender classification error rates between groups.

1270 images were chosen to create a benchmark for this gender classification performance test.

The subjects were selected from 3 African countries and 3 European countries. The subjects were then grouped by gender, skin type, and the intersection of gender and skin type.

Gender Labels

Gender was broken into female and male categories since evaluated products provide binary sex labels for the gender classification feature. The evaluation inherits these sex labels and this reduced view of gender which is a more complex construct.

The dermatologist approved Fitzpatrick skin type classification system was used to label faces as Fitzpatrick Types I, II, III, IV, V, or VI.

Then faces labeled Fitzpatrick Types I, II, and III were grouped in a lighter category and faces labeled Fitzpatrick Types IV, V, and VI were grouped into a darker category.

Three companies - IBM, Microsoft, and Face++ - that offer gender classification products were chosen for this evaluation based on geographic location and their use of artificial intelligence for computer vision.

While the companies appear to have relatively high accuracy overall,there are notable differences in the error rates between different groups. Let's explore.

All companies perform better on males than females with an 8.1% - 20.6% difference in error rates.

All companies perform better on lighter subjects as a whole than on darker subjects as a whole with an 11.8% - 19.2% difference in error rates.

When we analyze the results by intersectional subgroups - darker males, darker females, lighter males, lighter females - we see that all companies perform worst on darker females.IBM and Microsoft perform best on lighter males. Face++ performs best on darker males.

IBM and Microsoft perform best on lighter males. Face++ performs best on darker males.

IBM had the largest gap in accuracy, with a difference of 34.4% in error rate between lighter males and darker females.

IBM Watson leaders responded within a day after receiving the performance results and are reportedly making changes to the Watson Visual Recognition API. Official Statement.

Error analysis reveals 93.6% of faces misgendered by Microsoft were those of darker subjects.

An internal evaluation of the Azure Face API is reportedly being conducted by Microsoft. Official Statement. Statement to Lead Researcher.

Error analysis reveals 95.9% of the faces misgendered by Face++ were those of female subjects.

Face++ has yet to respond to the research results which were sent to all companies on Dec 22 ,2017

At the time of evaluation , none of the companies tested reported how well their computer vision products perform across gender, skin type, ethnicity, age or other attributes.

Inclusive product testing and reporting are necessary if the industry is to create systems that work well for all of humanity. However, accuracy is not the only issue. Flawless facial analysis technology can be abused in the hands of authoritarian governments, personal adversaries, and predatory companies. Ongoing oversight and context limitations are needed.

While this study focused on gender classification, the machine learning techniques used to determine gender are also broadly applied to many other areas of facial analysis and automation. Face recognition technology that has not been publicly tested for demographic accuracy is increasingly used by law enforcement and at airports. AI fueled automation now helps determine who is fired, hired, promoted, granted a loan or insurance, and even how long someone spends in prison.

For interested readers, authors Cathy O'Neil and Virginia Eubanks explore the real-world impact of algorithmic bias.

Automated systems are not inherently neutral. They reflect the priorities, preferences, and prejudices - the coded gaze - of those who have the power to mold artificial intelligence.

We risk losing the gains made with the civil rights movement and women's movement under the false assumption of machine neutrality. We must demand increased transparency and accountability.

Learn more about the coded gaze -algorithmic bias - at www.ajlunited.org

Dive Deeper:

Gender Shades Academic Paper

Test Inclusively:

Request external performance test

Request Pilot Parliaments Benchmark

McKinsey & Company Revolutionizes Customer Experience with Experience DNA Launch

January 25, 2021. NEW YORK & LONDON.

The new platform transforms how companies interact with their customers through actionable real-time insights.

McKinsey & Company today announced the launch of its new innovative data analytics platform, Experience DNA, to help organizations maximize the value of customer experience (CX) management and design. The first of its kind, the platform allows CX leaders to use real-time customer, operational, and financial data to make predictions about customer satisfaction, ultimately helping businesses target key customers.

It’s a significant leap toward real-time, comprehensive customer insight, giving companies a strategic tool to drive market differentiation by predicting the financial value for each customer segment.

Until now, organizations have been limited to using survey tools and sampling techniques developed nearly two decades ago. Only 6 percent of CX leaders believe using such historical insight to gauge loyalty enables both strategic and tactical decision making.1 These legacy tools are unable to translate today’s massive volume of data and signals into action to make faster, smarter decisions that drive acquisition, customer satisfaction, market differentiation, and profitability.

To meet the needs of the new customer, business leaders need to make disruptive changes to the customer experiences they offer. Experience DNA enables such change.

The New World of Act-Now Customer Experience

Experience DNA’s predictive insights tailor recommendations for strategic decision making and real-time engagement on customer journeys. Early adopters are seeing significant gains across a range of customer experience indicators, including:

• An airline company achieved an 800 percent increase in satisfaction among key customer segments through more-targeted post-delay compensation

• A hospital uncovered a $2.5 billion revenue uplift opportunity through improved segmentation

• A bank achieved a 20- to 40-point lower attrition rate through the identification of churn drivers

Experience DNA delivers a significant step change in customer experience insights and recommendations by combining data collection, aggregation, machine learning, and predictive analytics. This allows CX leaders to make informed decisions in the moment and in the future:

• Near 100 percent visibility of their customers and the impact of their experiences

• Make predictions on how business actions will impact satisfaction and value measures like revenue, cost to serve, or churn

• Drive real-time engagement decisions to improve an individual customer’s experience

• Take proactive action on opportunities and challenges

• Plan ahead on CX efforts to maximize both satisfaction and ROI, with value-based strategic decision-making and accurate performance measurement

Build Customer Relationships Through Proactive, Tailored Daily Engagement

It takes the average organization nearly a month to generate meaningful action on the results of a survey, and those actions are inevitably built for the “average respondent.” Experience DNA allows real-time insight and proactive intervention tailored to customers’ needs, removing the need to wait for a survey to identify an opportunity and enabling businesses to anticipate and respond immediately.

Prioritize Journey Enhancements and Drive ROI

Customer-centric organizations drive real value. In fact, since the last economic recession, CX leaders have seen threefold greater increases to shareholder returns than CX laggards.2 Experience DNA provides first-of-its-kind insight into the real experience drivers of financial value so organizations can prioritize actions that drive both customer loyalty and real ROI.

Kevin Neher, Senior Partner, McKinsey & Company

“2021 will be a year of transition as leaders look to pivot back to growth. It’s time for a bold reset to deliver experiences that are targeted, predictive, and responsive. By doing so, companies can go from ‘How are we doing?’ to ‘How do we deliver on what customers want, now?’ This customer-focused, data-driven approach allows companies to unlock efficiencies, predict customer behavior, track satisfaction drivers, anticipate churn, and act on real opportunities to create interactions that are truly meaningful and sticky.”

Nicolas Maechler, Senior Partner, McKinsey & Company

“While many corporate functions such as sales, operations, and supply chains have capitalized on new capabilities in data and analytics, CX has not leveraged these yet. Experience DNA enables CX leaders to leapfrog into the future by making CX predictive, real-time, and actionable in the moment. This new frontier of CX means that companies can utilize predictive scoring, foresee satisfaction, spot likely churn, and flag opportunities to turn loyalty into profitability.”

To learn more about Experience DNA, visit www.mckinsey.com/xDNA.

About McKinsey & Company

McKinsey & Company is a global management consulting firm committed to helping organizations create Change that Matters.

In more than 130 cities and 65 countries, our teams help clients across the private, public, and social sectors shape bold strategies and transform the way they work, embed technology where it unlocks value, and build capabilities to sustain the change. Not just any change, but Change that Matters—for their organizations, their people, and in turn society at large.

About Marketing & Sales, McKinsey & Company

The mission of the McKinsey Marketing & Sales Practice is to help leaders of both consumer and business-to-business clients create Growth that Matters through meaningful transformations and marketing-driven profit. The practice helps its clients set their strategic direction, develop their marketing and sales capabilities, and connect their organization to realize the full potential of today’s omnichannel opportunities. Clients benefit from McKinsey’s experience in core areas of marketing such as branding, customer insights, marketing ROI, digital marketing, CLM pricing, and sales and channel management.

For further information please contact:

Media Contacts: Lorena Duke / Jessica Mularczyk, Ascendant Communications, mckinsey@ascendcomms.net, +44 (0) 20 8334 8041 / +1 508 498 9300.

1 McKinsey Customer Experience Survey (survey of 260 CX leaders across 14 industries, completed in 2020)

2 CX leaders delivered threefold greater cumulative returns to shareholders from pre-recession in 2007 to post-recession in 2010 (Source: Forrester Customer Experience Performance Index 2007–2009)

Nielsen Study Reveals Significant Growth of Asian Representation on TV in 2021

May 18, 2022. NEW YORK.

Streaming Led with 11% Asian Share of Screen in 2021, Nearly Doubling Representation from 2020

A global leader in audience measurement, data and analytics, today released its 2022 Asian American, Native Hawaiian and Pacific Islander Diverse Intelligence Series report which noted a significant increase in Asian representation on screen in 2021. Across the top 1,500 shows in broadcast, cable and streaming video on demand (SVOD), SVOD led the way with 11% Asian share of screen compared to broadcast (3.2%) and cable (2.7%). The presence of Asian talent in top-rated shows like FBI, Equalizer, and Chicago Med, and the debut of Asian-led programs like FOX's The Cleaning Lady indicate the industry is responding to growing calls for more Asian-inclusive content.

"The media industry continues to make progress in its inclusive representation of Asian American, Native Hawaiian, and Pacific Islander (AANHPI) characters, themes and narratives," said Pat Ratulangi, Nielsen's VP, Diversity, Equity & Inclusion. "However, that representation on screen is still below the Asian population in the U.S. Now is an important time for the industry to highlight Asian characters, stories and experiences on screen through culturally inclusive programming. Accurate representation on screen can lead to greater understanding, inclusion, engagement and peace off-screen."

In the last two years, AANHPI representation in streaming has almost doubled from 6.1% in 2020 to 11% in 2021, the report found. This improvement, coupled with Asian American consumers' growing hunger for more authentic and representative content (two-thirds of Asians still feel there is not enough representation on TV), present an opportunity for more shows that highlight the richness of Asian American experience across platforms.

Overall, Asian representation across broadcast, cable and SVOD increased to 4.6% in 2021 (up from 3.5% in 2020). The report notes a significant improvement in representation in the top 10 most-watched shows on broadcast and cable. In 2021, half of the top 10 programs had some Asian talent representation, compared to 2020 when none of the top 10 most-watched shows did. Asian women were present in three of those shows (NCIS, Equalizer and Yellowstone) and Asian men were present in two (Chicago Med and FBI).

Other key highlights in the 2022 Asian American, Native Hawaiian and Pacific Islander Diverse Intelligence Series include:

• Streaming programs drove Asian multigenerational co-viewing. Gen Z co-viewing on Netflix was 4.3x higher than audiences overall. On Netflix, for example, 8.4% of Asian American viewers aged 18-24 watched with someone aged 65-74.

• In 2021, there was a greater diversity of themes in shows with Asian representation — such as friendship, teamwork, and creativity — than in 2020 when the dominant themes were more stereotypical – cerebral, thoughtful, and good.

• More than half of Asians surveyed said they are more likely to buy from a brand that advertises in shows that feature Asians, creating an incentive for advertisers to engage with this population.

• A quarter of brands invest just 4% or less in the programs that represent Asians at parity, while the leaders for Asian-inclusive ad spending invest almost 10x as much.

Download the 2022 Asian American, Native Hawaiian and Pacific Islander Diverse Intelligence Series for more details and insights. Please visit www.nielsen.com/asian-american to learn more. Join the discussion on Facebook (Nielsen Community) and follow us on Twitter (@Nielsen_DEI).

ABOUT NIELSEN Nielsen shapes the world's media and content as a global leader in audience measurement, data and analytics. Through our understanding of people and their behaviors across all channels and platforms, we empower our clients with independent and actionable intelligence so they can connect and engage with their audiences—now and into the future.

An S&P 500 company, Nielsen (NYSE: NLSN) operates around the world in more than 55 countries. Learn more at www.nielsen.com or www.nielsen.com/investors and connect with us on social media.

View original content to download multimedia:https://www.prnewswire.com/news-releases/nielsen-study-reveals-significant-growth-of-asian-representation-on-tv-in-2021-301550193.html

SOURCE Nielsen