Covering Disruptive Technology Powering Business in The Digital Age

I’ve recently taken on a position as an executive in a technical role at a big data startup headquartered in the heart of Silicon Valley.  I see the growth of next-gen big data platforms as the most exciting and disruptive story of the 21st century.  The entire computational landscape is going through a profound change; legacy technologies are in decline and a new vanguard of distributed data platforms are emerging as the next-gen infrastructure for enterprise and research applications.  It’s a great time to be a technologist.

But it’s hard not to notice that in the midst of all of this new growth and innovation, women have played a decidedly small role.  This isn’t a new phenomenon; the gender gap in technology companies and in computer science degree programs has been well documented.  But since a lack of gender diversity is a headwind for any industry, it bears further exploration into the root causes.

Are Women Just Bad at Science and Math?

Why is it that women aren’t as numerous as men in technology and computer science?  Ten years ago, Larry Summers famously questioned whether the relatively few women in top positions in science might be attributed to “issues of intrinsic aptitude.”  Despite the public outcry that Mr. Summers understated the role of discrimination and socialization in the process of attrition, it’s hard to fault someone who merely asks the question from a perspective of curiosity.  I’ve even asked myself whether it’s possible that there is an innate difference in the abilities of women and men to reason analytically.  Do we have evidence that could prove or disprove this theory?

Personally, I have come to the conclusion that women are equally capable as men in science, technology, and math.  I believe this largely because it resonates with my own personal experiences collaborating with women and men in the field.  But there is also scientific evidence that, in my mind, makes it clear that proficiency in science and math is a result of interest and motivation — not innate talent.

For example, in 2005 the Organization of Economic Cooperation and Development administered a standardized test in 41 countries and found that among 15-year-old Icelandic students, boys trailed far behind girls in math aptitude.  A subsequent analysis by Icelandic researchers found that the discrepancy owed to differences in motivation and career ambition; boys planned to become fisherman and saw math as irrelevant whereas girls knew that a non-fishing career would require an education and admittance to college.  Similar studies have identified other non-Western cultures where there are equal numbers of female and male computer science students, such as in Malaysia. In each study, the researchers explain these numbers by pointing to reasons of motivation and how the discipline is perceived by students of either gender.

The Motivation Gap

If proficiency in science and math is correlated with motivation, then clearly many women are not, by and large, motivated to go into such fields.  There must be a psychological barrier that causes women to, in effect, opt out of careers in science and math.

It starts in school.  A number of compelling studies by researchers at Carnegie Mellon University and others describe the psychological barriers that cause female students to opt out of computer science degrees.  In short, the rationale for opting out of computer science as a chosen major owes to perceptions that: 1) it’s too hard, 2) it’s boring, or 3) I won’t fit in with the other CS students (who are predominantly male and sometimes can be a little antisocial).

Perceiving computer science as an unpleasant discipline and technology as an uninteresting career path in turn causes women to suffer a lack of motivation in their math and computer science courses starting in secondary school and college.  In fact—and this goes contrary to the popular wisdom surrounding the story of Silicon Valley’s gender gap–these perceptual barriers to entry are much more important in explaining the attrition of women from science and technology careers than gender discrimination, unconscious bias among hiring managers, or a proverbial “glass ceiling”.

Is This Really a Problem?

If we concede that women are fleeing from the tech industry of their own volition and not so much because of gender discrimination, is it such a bad thing that women are underrepresented in tech and big data?  Well, yes, it’s a problem for big data and other technology companies that need to hire exceptional talents when nearly 50% of the population is choosing to steer clear of our entire industry.  And it’s worse for the country because the tech industry happens to be a key driver of the growth of US GDP.

A Path Forward

The good news is that perceptions can change.  In my previous role as a professor of computer science, I worked with my colleagues to change perceptions around our field in order to increase the ratio of women enrolling in our CS courses with tremendous success (with women now comprising more than 50% of the pre-registered students in every single CS class for the upcoming fall 2016 semester).

The changes we implemented were straightforward.  Namely, we streamlined the students into different classes so true beginners would not be intimidated by those with more experience; we gave students options for assignments with choices ranging from applications from computational biology to sports analytics so the material was more engaging; and we made an institutional commitment to take students at all levels to the Grace Hopper Celebration of Women in Computing in order to provide a networking opportunity to connect with other women in the field.  Each of these three strategies addressed one of each of the problematic perceptions of computer science and in total they had a dramatic impact in just two years.

So I’m optimistic.  I fully expect the ratio of women entering big data and other tech subfields to grow in the next generation.  And in the meantime, I can offer young women technologists a few key tips in order to thrive in this fast-paced and exciting field.

Keys to success for women in big data:

• Work hard. Big data is an incredibly competitive field for both companies and employees.  Being smart is a prerequisite for entry into a big data company—smarts are just table stakes.  The way to differentiate yourself is through hard work, consistency, and discipline.  (This is true for men too, by the way.)
• Speak up. Your opinion counts.  The value in having a diverse team is that you come to a problem with different viewpoints, ensuring you examine the issue from all angles.  A good manager wants to work with people who know things he or she doesn’t already know – not people who just mimic back what they heard from their peers.
• Be yourself. Don’t put too much time or energy into trying to “be one of the guys” in order to fit in.  It’s a waste, and if the effort isn’t sincere, you will come off as phony.
• Look for mentors and allies. Everyone who has done anything important can point to the set of people who helped them learn the ropes and then grow into their fullest potential.  Align yourself with people who invest time and energy into you, and keep in mind that these people may very well be men.

In time, as perceptions about computer science change, we will start to see the profile of the big data community shift.  When the next generation sees that industry participants need not all be male uber nerds, but can instead represent a broader segment of the population, the result will be a much more diverse group of industry entrants, including a healthy number of women.

This article was originally published on www.datanami.com and can be viewed in full

(0)(0)