Algorithms Have Nearly Mastered Human Language. Why Can’t They Stop Being Sexist?

Abel — Fri, 20 Sep 2019 20:54:32 +0000

Teaching computers to understand human language used to be a tedious and imprecise process. Now, language algorithms analyze oceans of text to teach themselves how language works. The results can be unsettling, such as when the Microsoft bot Tay taught itself to be racist after a single day of exposure to humans on Twitter.

It turns out that data-fueled algorithms are no better than humans—and frequently, they’re worse.

“Data and datasets are not objective; they are creations of human design,” writes data researcher Kate Crawford. When designers miss or ignore the imprint of biased data on their models, the result is what Crawford calls a “signal problem,” where “data are assumed to accurately reflect the social world, but there are significant gaps, with little or no signal coming from particular communities.”

Siri, Google Translate, and job applicant tracking systems all use the same kind of algorithm to talk to humans. Like other machine learning systems, NLPs (short for “natural language processors” or sometimes “natural language programs”) are bits of code that comb through vast troves of human writing and churn out something else––insights, suggestions, even policy recommendations. And like all machine learning applications, a NLP program’s functionality is tied to its training data––that is, the raw information that has informed the machine’s understanding of the reading material.

Skewed data is a very old problem in the social sciences, but machine learning hides its bias under a layer of confusion. Even AI researchers who work with machine learning models––like neural nets, which use weighted variables to approximate the decision-making functions of a human brain––don’t know exactly how bias creeps into their work, let alone how to address it.

As NLP systems creep into every corner of the digital world, from job recruitment software to hate speech detectors to police data, that signal problem grows to fit the size of its real-world container. Every industry that uses machine language solutions risks contamination. Algorithms are given jurisdiction over public services like healthcare frequently exacerbate inequalities, excusing the ancient practice of shifting blame the most vulnerable populations for their circumstances in order to redistribute the best services to the least in need; models that try to predict where crime will occur can wind up making racist police practices even worse.

by Abel Ayuba

“As a software engineer, your main contribution to the project isn’t the code that you write”

Abel — Tue, 27 Aug 2019 15:54:31 +0000

“As a software engineer, your main contribution to the project isn’t the code that you write”

Your main task is to shape the way that the software is designed, built, and tested, such that all subsystems work well together, and, eventually, run smoothly in production. This requires a bird’s-eye view across the boundaries of the different project phases and a critical attitude towards the processes, architecture, organization and technology that eventually all have an impact on the quality of the software that’s produced. Of course, you’ll spend considerable time coding. That’s the core of software engineering. However, that time will be unproductive if you don’t spend enough time knowledge sharing, setting the standards for design and documentation, and optimizing the toolchain that’s used for CI/CD.

“For a software engineer, the real testing takes place in production”

When we say the real testing takes place in production, we don’t mean to state that the code shouldn’t be properly tested upfront. It goes without saying that strong test automation should be conducted. However, automating these tests can take up a lot of time and money, so it’s important to initially determine the criticality of your application. You simply can’t spend unlimited resources on every application writing test automation. So, it’s important to balance customer cost with customer value and timebox – with your project team and customer – how much time we agree is feasible to spend on automated testing, and how much time we’ll allow for ‘real-life testing’ and quick fixes while in production.

In practice, automated tests can filter out many bugs, but the most complex situations emerge in the wild: when the code runs in production. Consequently, if you’re striving for a high-quality application, you should make sure that you can quickly react to errors when your code runs in production. That’s why a DevOps approach is essential. Through constant monitoring and logging of incoming data, unexpected behaviors can be spotted early on and the team can act on these before they become a real problem. DevOps allows for short development and test cycles, making your applications flexible for change and improvement. In other words: if your way of working is agile and you’re prepared for rapid change cycles, testing in production can be an agreed optimal way to go.

“As a software engineer, your job is to make yourself redundant”

This means that the work you deliver should have such quality that future work is either automated away or transferable to a colleague at any time. Of course, it sounds controversial to work in a way that could potentially leave you without any work but this also means your employer and/or customer can be flexible in using your skills in different projects and solutions. Any sudden need for your expertise in another project, any shifting in priorities – which moves current work to the backlog – or even any sudden leave or absence shouldn’t prevent a colleague from picking up where you left off.

What does this mean for your daily activities? Besides communicating and documenting the decisions you make, the software you deliver should be thoughtfully and understandably designed and crafted. It should be clearly written and documented, it should have a consistent style, formatting and syntax, and it should be properly structured and versioned. In short: this means writing well-documented and easily readable code. The better you become at being redundant, the more valuable you become to your employer or your customer.

Forem: Abel

Algorithms Have Nearly Mastered Human Language. Why Can’t They Stop Being Sexist?

“As a software engineer, your main contribution to the project isn’t the code that you write”