Reading 29: Ethical Software Engineering
Software in 6.031
Objectives
After this reading, you should be able to:
- Explain the ethical principles to consider when you design, build, and maintain software.
- Apply four different moral lenses as you examine the ethical ramifications of a particular system.
SFB, ETU, and RFC
The three technical objectives for software in 6.031 have appeared at the top of every reading and in the discussions during every class this semester. There are many other properties of good software that we have not discussed (performance, security, usability, …) that may be as critical to the success of a software project as correctness. The topic of this reading is another such property: ethicality.
Your practice of software construction, whether as a professional software engineer or in one of the innumerable other positions where you might write software, does not take place in a vacuum:
- You work with other people
- on software designed by other people
- that is used by other people
- whose use of your software affects other people.
Writing software that is correct, clear, and changeable is your professional obligation to (1) the others you work with and (2) the organization you work in. Correctness is also an obligation to (3) the users of your software, who expect it to function as described. Expanding outward from those users, we encounter both (4) the other people who are affected by your software, even if they don’t use it themselves; and we must consider the wider world in which all those people live.
We’ll start, however, with some ethical principles around writing code, before we go on to consider larger questions.
Plagiarism, copyright, and licenses
One ethical issue you’re familiar with is plagiarism — we asked about it on Problem Set 0. The Academic Integrity at MIT handbook says: “honesty is the foundation of good academic work.” The 6.031 collaboration policy and the handbook section on Writing Code describe how plagiarism violates that obligation to honesty. Any time we use code written by others, we must attribute it to the authors and give them credit for their work.
The 6.031 collaboration policy also describes when and how sharing your work from this class is and is not allowed. Software engineering has a strong history and culture of sharing. The free software movement considers unethical any software that users are not free to run however they want, modify to suit their needs, and redistribute in both original and modified form. The open source movement advocates for sharing source code as a practical advantage over keeping it proprietary. Many pieces of software that provide the foundation for modern computing are free software (meaning freedom-preserving, not simply $0). The culture of sharing is also visible in places like Stack Overflow, where programmers have collaboratively created an enormous wealth of explanations and examples from which we all benefit.
reading exercises
Alice, Bob, and Charlie are working on the 6.031 group project. Danielle is their TA mentor. Prof. Elliot is teaching the class.
Alice and Bob work together to implement a piece of the project that Charlie never touches. They receive advice from Danielle about how to make the code more SFB.
According to the 6.031 collaboration policy, who owns the copyright to that code?
(missing explanation)
A code of ethics
As our next step for understanding ethical software engineering, we will use the Association for Computing Machinery (ACM) Code of Ethics and Professional Conduct. The ACM is the leading international professional organization for computer scientists. (Among other things, the ACM gives out the Turing Award, the CS equivalent of a Nobel Prize.)
General ethical principles
Read:
ACM Code Section 1, General Ethical Principles
Ethical structures, not just ethical individiuals
Section 2, Professional Responsibilities, provides a list of obligations you could consider not only as an individual, but when you join an organization. For example, you should “strive to achieve high quality in both the processes and products” of your work (2.1) and “maintain high standards of professional competence, conduct, and ethical practice” (2.2). But these are not goals you can achieve alone.
When large organizations build a system that is detrimental to human well-being — it causes harm, it discriminates, it increases inequality, it violates privacy — it might be the case that some single bad actor was the evil mastermind behind a nefarious plot to hurt people. If that’s the case, that bad actor probably is an actor, because you’re probably watching a movie where the storytelling is driven by individual villains and heroes.
Much more likely is that the large organization builds a harmful system because it doesn’t have a structure in place to ensure that the collective effort of individual participants has an ethical result. Worse, because we live in a world and society where so many existing structures are ethically unsound — they discriminate, they perpetuate inequality, they deny autonomy, etc. — an organization that pays no particular attention to acting ethically will end up reproducing in its work the ethical failures of society around it. (See, e.g., Prof. Susan Silbey on How Not to Teach Ethics.)
When you interview at a company, you might ask: “how do you do code reviewing?” With that question, you’re probing one part of how that company creates a high quality software process (2.1), shares technical knowledge (2.2), and provides professional feedback (2.4). You should also ask how the company understands and reviews the impact of its projects on people, because merely hiring good employees and setting out on a good path is not enough. If the company hasn’t designed its own structure so that ethical questions are asked and answered and the objectives of whole teams and projects are updated in response, you should be as skeptical as if your interviewer says: we don’t need code review, we trust every engineer to write great code on their own.
With great power…
Finally, Section 3, Professional Leadership Responsibilities, describes ways that leaders should build ethical processes into their organization.
In explaining the general principle to avoid harm (1.2), the Code references an item from section 3: “recognize and take special care of systems that become integrated into the infrastructure of society” (3.7). Wouldn’t we all love to build a system so wonderful and widely used that it becomes a part of everyone’s everyday lives? Perhaps you can think of some such systems (social networks, again, are the ready examples). The builders of many of today’s widely-used software systems absolutely did not recognize the need for special care. As the rate of adoption of their products went up, so too did their ethical responsibilities. They failed to anticipate and design for the societal implications of their systems. The next section will offer a procedure that you and your world-changing startup can use to do better.
Moral lenses
“Professional competence starts with technical knowledge and with awareness of the social context in which their work may be deployed. Professional competence also requires skill in communication, in reflective analysis, and in recognizing and navigating ethical challenges.” (ACM Code 2.2)
How can we begin to do that recognizing and navigating?
Read:
Moral Lenses (MIT certificate required)
This reading describes a process for ethical evaluation. It applies four moral lenses, different viewpoints on a project’s positive and negative impacts:
- Outcomes
- the project’s costs and benefits
- Process
- the way they are achieved
- Structure
- patterns of good and bad outcomes and processes
- Character
- considering the project as a person
The practice exercises at the end are for you to consider on your own.
reading exercises
Software beyond 6.031
Correct, clear, changeable software is a starting point. Without correctness, virtually no other desirable property is achievable, because we can’t rely on the code to do what we intended. And without clear, changeable code, multiple authors will be unable to collaborate, and even a single author will eventually lose the battle with complexity and be unable to build further.
Although we don’t address them in 6.031, properties like performance, security, and usability cannot be bolted on later: they must be baked into the system as part of its iterative design process.
Ethicality is no different. Wherever you go on to build software, consider and test its ethical properties just as you would any others. Do not rely on yourself or anyone else to simply “get it right.” We rely on testing to tell us whether our implementation successfully satisfies a spec, and when we put on our testing hat, we work as hard as we can to break our own code. With your ethical testing hat, consider as pessimistically as possible all the ways that what you build might be misused and abused, using the moral lenses from this reading. Fixing these bugs might be tricky, because they involve so much more than just the system and its source code. Luckily, you have plenty of experience fixing tricky bugs.