User education and paternalism
Don Marti agrees with several observers that user education is not the answer to computer security problems -- contrary, I think, to a commonplace view only a few years ago and perhaps still today.
As editor of Linux Journal, Don published an article I wrote called "Give TCPA an Owner Override", in which I worried about security technologies that irrevocably take choices away from end users. Interestingly, there are (at least) three threads among trusted computing advocates and implementers, only one of which is particularly relevant here. The first two threads are about allowing or forcing users to give up certain kinds of control in on-line interactions because the user might do something that a publisher or service provider doesn't like, or that other users wouldn't like. There is then an argument about whether this is good for the user and whether it's important whether it's good for the user or not. The third thread is about allowing or forcing users to give up control because the user might do something that the user would regret but that the user wouldn't be able to understand was wrong.
This falls, for example, under the heading of protecting users against worms and Trojan horses that might undermine security policies that are thought to be in the user's pure self-interest. Microsoft gave a trusted computing demo that emphasized that NGSCB (in its original design) could help protect users in a way that I have called paternalistic. This paternalistic approach to security is related to the traditional concept of mandatory access control; NIST, for example, writes that MAC is employed where
the security policy of a system dictates that:
- protection decisions must not be decided by the object owner.
- the system must enforce the protection decisions (i.e., the system enforces the security policy over the wishes or intentions of the object owner).
Trusted computing systems with remote attestation provide a means of implementing a kind of mandatory access control in a PC operating system for paternalistic reasons: because the user is at risk of making bad decisions (under the influence of phishing attacks, to take just one example, or if the user voluntarily installs dangerous spyware, to take another) and therefore must not be permitted to defeat or circumvent the security policies.
Some recent informal conversations with TCG members persuaded me of three things. First, it is actually possible to implement some kinds of paternalistic security under TCG. For example, you can give users an authentication credential that they can't transfer or give away, that spyware can't steal, and that the users can't even authorize spyware to steal if they wanted to. That's pretty interestingly and pretty different from the status quo. Indeed, you can build a client that allows people to access a particular service (say, a bank account or an on-line auction) in a way that actively prohibits them from delegating or transferring any of their authority to another person or noninteractive process, even if their system is compromised, or even if they want to transfer it. (Well, there are limitations to this; they could still tell another person their passwords, but those passwords would never work if used from a different client machine, because the server has a way to distinguish different clients, or at least to distinguish different identity-to-client bindings.)
Second, some people consider this ability a prerequisite for some kinds of on-line commerce because they simply do not believe that high-value transactions can safely be conducted on traditional general-purpose operating systems on general-purpose computers without an attestation feature.
Third, some people who aren't on board the DRM bandwagon but who come from traditional computer security think that this paternalism or mandatory access control application is important enough by itself to justify trusted computing deployment.
Now, the tricky thing for me, and I presume for Don, is that this paternalism is really paternalistic paternalism. It is the genuine article. The way you enforce a policy like "the user has a credential that can't be transferred to spyware even with the user's consent" without being psychic or AI-complete is that you define every single piece of user-installed third-party software as presumptively spyware. That is, you say that the user has a credential that cannot be transferred by the user to any program (sc. "operating environment", "software stack") that is not on a pre-approved list. And you have technological enforcement of this; you have mandatory access control; it doesn't matter what the user thinks. If the user modifies the local software environment or uses an unrecognized client, it temporarily voids the user's authorization to authenticate to particular services. And this does away with the need for user education for security for those servics, because the user is physically incapable of doing anything that will violate their security policy!
Now, some computer security people are saying that this is, in some form, the only plausible future for computer security. I have heard a sober argument that we need to be able to have a sort of formal proof that the user is not running a program that is against the user's own interest, or that would expose high-value transactions (whatever you want to define them as) to surveillance or interception or falsification by an unauthorized party. And of course the only way to do this is to say that the user shall not run certain programs for certain purposes, at all ever, whether the user wants to or not. This is not quite the same as an application blacklist or a document revocation list or not having a general purpose computer, because you can run whatever you want when you're not doing something "security-sensitive"; you could, in one extreme, have an operating system full of viruses and spyware and games, and another operating system full of financial software and VPN clients and secure remote desktop tools and cryptographically-enabled e-mail clients, and just not let them interact at all. (Or, continuing the MAC analogy, the financial software could read the viruses and spyware and games, but not vice versa.)
In this model, the user gets to decide what software to run in general, but not what software to run while doing something security-sensitive. One interesting question here is how we know that people actually implementing security software will have the knowledge or the incentives to act in the user's interest and not in some other interest (and a related question is about what happens when there is a merger of pro-user and anti-user features in a single program, or a different mix of features in rival programs, or a conflict of interest, or the user actually does know better in a particular case but the security policy author doesn't have the knowledge or the authority to agree that the user really does know better). Another interesting question here is whether things like Internet access will eventually fall into the category of things that are considered security-sensitive; cf. Trusted Network Connect, which lets ISPs define what client software configurations are acceptable, a power they've arguably never had before and which many of them will claim to be exercising in the end-user's interest.
While I agree with the sentiment that there is much to be done in security and in security usability, I am nervous about the entire abandonment of the concept of user education. The only possible consequence seems to be building systems that don't (for fear of phishing and the like) even permit users to do things that are against the security policy and that give them no way to override or alter it.
We were once told (in a sentiment attributed to Doug Gwyn) that "UNIX was not designed to stop you from doing stupid things, because that would also stop you from doing clever things" and it seems clear that at a certain level this is an inherent tension. If users can (without giving up interoperability!) debug their kernels, then they can install kernel rootkits, even at the request of persuasive phishers. If users can replace their browsers, or install browser plugins, then they can install spyware. If users can install screensavers, then they can likely install keyloggers. If users can script their operating systems, then they can run script kiddies' packages on their operating systems. (N.B. There are, of course, operating system security techniques that will mitigate some of these risks. But that brings us back to the same problem: if users can alter the implementation of the security techniques that mitigate the risks, they can counteract the mitigation of the risks and reintroduce the risks...) On the other hand, if someone has a technical mechanism for blocking spyware, for blocking certain third-party applications, for allowing only a single application ever to read a particular resource, for forcing software upgrades, and so on, then there is a political question about how these powers will be used, and with what collateral damage to tinkering values and open-standards values. I have also alluded to this essential tension in section 2, and especially section 2.3, of the EFF Comments on the TCG best practices document.
The article by Jakob Nielsen that Don cites does not seem to propose particularly paternalistic steps; it proposes mostly steps related to usability. However, Nielsen does not suggest creating security measures that users can't defeat or override; that means phishers can still try to persuade users to defeat or override these measures. Clearly some kind of user education would still be necessary, wouldn't it?
If we really want a computer security in which users can never be tricked into violating the security policy at all, we need to take users' unpredictable, not-characterized-by-formal-models subjective decisionmaking processes out of the loop. I see a long-term trend headed in that direction, and I worry about it. The trend seems to involve replacing user education by paternalism that includes enforced limits on users' software choices, at least in particular contexts. And it is more than a little tempting to accept this in a few contexts while fearing what lies at the end of this path. I would like to know what Don has to say about this trend and how it relates to his skepticism of user education.
