GUI Design for Power Users

[Crazed123]

Now we all know that WIMP user interfaces seem to work fairly well for new users of any system -- sit them down with a tutorial, a small manual and the machine and they do fine.

We also know (as common knowledge) that power users can more efficiently use the machine through some kind of command-line interface, or even a scripting language.

But in modern systems, many applications can only receive limited options and parameters at the command line, and GUI programs may take little or no input from their standard-input handles once they've begun.

So we end up with the question, how can we design a GUI system that will work for power users? First of all, my basic list of criteria for "power user" traits:

1.They know what they're doing, and just want to get it done. This explains why power users hate Clippy.
2.They may or may not want to accomplish their goals through scripting. When they haven't automated their actual task via scripting, they often want to build the specification of the new task (like a command-line string) out of scripted parts. They also want to mix scripts and programs when necessary.
3.They usually find using the keyboard more convenient than using the mouse, because it has a larger number of input "digits" accessible with greater speed and precision. Using the keyboard for non-text input (or encoding control and command inputs in text) also removes the need to stop typing and move a hand over to the mouse.
4.When they don't know what they're doing, they want a convenient and consistent way to access documentation that will tell them what they're doing. The Unix "man" command provides a good example of this.
5.They want to access programs via a consistent interface. Why learn multiple, conflicting sets of shortcut keys for the same functions in different programs?

I have two basic ideas on how to handle this issue. One (before Solar can post it) is to bring back something like ARexx from the Amiga -- a scripting language to which applications can add arbitrary new commands for use by users. This would solve 1-3 and 5 by providing a consistent scripting interface to all programs, including those running under a graphical interface. If programs implemented ARexx ports accessible by IPC, users could also issue script commands to an already-running program instead of passing the script to a new instance of the application.

To solve #4, I propose adding a shortcut key, defined by the window manager, that will issue a command to obtain and display documentation based on a textual or graphical (ie: "What is this?" help from Windows) designation of the item in question. This solution would also enhance #5 by providing a single shortcut key useful in every program.

I also think we should add widgets to our windows (all of them) to control input focus. A power user may want the ability to browse the web with his mouse while typing commands into a command-line shell with keyboard focus. The user will explicitly control which window receives the input from which device, rather than allowing the window manager and applications to fight each other for focus against the user's wishes.

Finally, we could just replace our mice with knock-off Wiimotes from Hong Kong. This would provide valuable input data for navigating the 3D user interfaces of the future.

Does anyone have any interesting ideas of their own, or opinions on my ideas? Go ahead and post; I can't steal your ideas since I haven't even reached the GUI stage yet!

OK, that's a Theory/Design topic. Let's see how many people take interest.

[Brendan]

Hi,

Some minor notes...

When a power user doesn't know what they're doing, they're learning.

The same GUI should be usable by power-users who know what they're doing, power-users who are learning and non-power users.

Non-power users and power users who are learning need documentation about how to use the software directly (which widget does what). Power users who know what they're doing need documentation for the scripting interface. IMHO this is 2 completely different sets of documentation. Both types of documentation should be presented in an interactive way (i.e. the "What is this?" help from Windows) and in a way that can be printed on paper (i.e. like "man" in *nix).

There should be a "record" button, so a user can use software directly (clicking buttons, selecting things from menus, pressing keyboard shortcuts, etc) and record those actions as an auto-generated script, where running that script repeats the actions they did manually. This is good for quickly creating simple scripts and for learning the scripting language.

All applications should accept the "--hidden" command line argument that starts the application in a special mode where no window is created. This means if you run a script that uses several different applications you don't end up with the screen going insane as applications are started and stopped (but you could omit this argument, so that you can write a script that launches an application and does some stuff in advance before the user takes over).

The same scripting language should be used by the entire OS, so that it can be used for shutting down the computer, creating directories, installing applications, backups, creating dialog boxes and windows, controlling sound, accessing MySQL, using file I/O and sockets, spawning threads and processes, etc.

There should be an interpretter/debugger for the scripting language, so people can step through a script and stop it at any time and view internal variables, etc.

There should be a compiler for the scripting language, and people should be able to write normal applications and games using the scripting language (and sell the resulting binaries like any other commercial software if they wish).

Eventually, the OS itself should be ported to this scripting language, and then the scripting language should be ported to other architectures.

I'll let you decide how much of this is serious and how much is just for fun - to be honest I'm not too sure myself...


Cheers,

Brendan

[Colonel Kernel]

@Crazed123:
You brought up many interesting points, but sadly I only have time to respond to a few of them.

I don't have any revolutionary ideas about replacing WIMP with something else, but I have realized some things about psychology and how it relates to good versus bad GUIs.

First, Clippy -- I've come to realize that what I find so offensive about Clippy is the same thing that bugs me about the little notification bubbles that pop up in Windows, and about pop-ups in general: I want to tell my computer what to do, and I want to have it do it. I do not want to have a conversation with it. I would imagine this is true for all power users. "You're my computer, dammit, just do what I tell you! Don't talk back!" This should be an overriding design principle for a "power user UI", or any good UI for that matter. People get depressed and angry when they feel they aren't in control of their lives. I think this also applies to UIs.

Second, on what it means to be a "power" user. I believe that it is possible to appeal to power users without necessarily becoming more keyboard-centric. I've had two experiences recently that support this, since switching from Windows XP to OS X. The first was when I started using Exposé to switch between windows. Being able to see all my window contents at once just by squeezing the sides of my mouse made me feel "empowered". I know that sounds very corny, but it's true. Fiddling around with tiling windows and alt-tab on XP feels like digging a trench with a spoon compared to Exposé (I haven't tried Flip3D in Vista yet, although it looks like only part of the window contents are visible). The second was when I migrated everything from my Mom's old PowerMac to her new iMac. I set up a network share and just started dragging stuff across, including installed applications and preference files. Never mind that one box was PowerPC and one was Intel, that one was running OS 10.3.9 and one was running 10.4.8, etc. Everything just worked -- no keyboard involved. My point is, the system as a whole was obviously well thought-out, and eliminated the need for a lot of fiddling around with installers, settings, etc. With a few mouse drags I accomplished what would have taken me hours if I were migrating between two Windows boxes.

I know that sounded like a big Mac ad, but before you start flaming me, remember it was just an example. To summarize, the principles that underlie "powerful" UI design include:

• Have the UI just do what it should do, and not ask too many questions or interrupt the user.
• Use visual effects in a way that makes some routine tasks much easier, not just for eye-candy.
• Make the system conceptually simple so that it's far easier to do things through your UI than it would be on other systems.

I believe than an enormous amount of complexity in modern UIs is accidental, not incidental, complexity. IMO this wouldn't be the case if more thought were given to the little things that appeal to human psychology rather than to try to distinguish "power users" from regular users.

-edit-
I just read Brendan's post (I type too slow ) and figured I should mention re: scripting, that I have not tried Automator yet. If and when I do, I'll post about my experiences with it. I haven't written any AppleScript either...
-edit-

[Crazed123]

Hi,

Some minor notes...

When a power user doesn't know what they're doing, they're learning.

The same GUI should be usable by power-users who know what they're doing, power-users who are learning and non-power users.

Non-power users and power users who are learning need documentation about how to use the software directly (which widget does what). Power users who know what they're doing need documentation for the scripting interface. IMHO this is 2 completely different sets of documentation. Both types of documentation should be presented in an interactive way (i.e. the "What is this?" help from Windows) and in a way that can be printed on paper (i.e. like "man" in *nix).

Agreed. A good GUI for anyone needs discoverability.

There should be a "record" button, so a user can use software directly (clicking buttons, selecting things from menus, pressing keyboard shortcuts, etc) and record those actions as an auto-generated script, where running that script repeats the actions they did manually. This is good for quickly creating simple scripts and for learning the scripting language.

In practice this means that a recorder program (perhaps in the window manager?) must lock input focus, log all inputs, and forward them to whichever program should actually receive them. Programs could also emit some kind of script forms to a special "standard scripting" output, which a recorder could read.

I personally like the second approach better, as it doesn't make keyloggers hyper-convenient.

All applications should accept the "--hidden" command line argument that starts the application in a special mode where no window is created. This means if you run a script that uses several different applications you don't end up with the screen going insane as applications are started and stopped (but you could omit this argument, so that you can write a script that launches an application and does some stuff in advance before the user takes over).

Got any way to make all applications conform to your standard? It's a good idea, but I don't see how the system can enforce it.

There should be a compiler for the scripting language, and people should be able to write normal applications and games using the scripting language (and sell the resulting binaries like any other commercial software if they wish).

Eventually, the OS itself should be ported to this scripting language, and then the scripting language should be ported to other architectures.

Your idea runs into some implementation trouble here. The main point of a shell or scripting language is to treat full-fledged programs (in binary or in another interpreted language) as simple subroutines of the script.

However, an operating system also has the imperative to protect programs from one another. So you can't have a scripting language capable of declaring a pointer to a program's internal memory, because the operating system runs the script interpreter (or compiled script) in a separate protection domain (usually a separate address space) from the invoked program. And that requirement would cripple the language for writing the operating system itself.

Scripting languages also usually need access to named persistent storage as part of their runtime (for fetching those subroutine programs), and you can't write a kernel that requires file-system access.

Finally, how would such a scripting language approach concurrency?

However, this could all work if you came up with a method of protection that allows scripts to access internal program memory in a protected fashion. Performing the protection checks to determine if one routine had the requisite privileges to use arbitrary pointers might add a bit of overhead at run-time, but you could do it.

Once you added inline assembler and the ability to link to assembler routines (for interrupt stubs) to such a language and figured out how to make it allow concurrent subroutine continuations, you could write an operating system in your ideal scripting language. Everything would eventually get compiled down to the scripting language, which you would proceed to compile to binary or byte-code if you so desired.

EDIT: You would also have to add a mechanism to your operating system for linking new top-level functions (programs) into the global function name-space at runtime. This would mimic the way in which adding a program to /usr/bin/ makes it show up in bash's program name-space.

I set up a network share and just started dragging stuff across, including installed applications and preference files.

I'd like to pick up on this and say something: not only should GUIs have simple primitive functions, but good GUI scripting languages (as Brendan and I have talked about) should include support for interpreting input from any device. If a system doesn't come with select-drag-and-drop support, the user should be able to add it.

[mystran]

I would consider myself a power user, and I personally find myself most confortable with Gnome nowadays: minimalism without anything extra, yet still flexible enough that if you want to change something, it's possible.

I use Nautilus in the gnome default spatial mode, where each folder is a window, with menu and statusbar but nothing extra. If you try to open an open folder again, it'll just raise the old one. Each folder remembers all it's settings, including position and size. Most options dialogs only have "close" button, because the option becomes live as soon as you change it (there is no need to apply anything). Absolute newbies survive with my desktop just fine. But specifically because there is little extra, there is little to get on my way. And a terminal is always one click away anyway, though I use it surprisingly little unless I'm programming..

As for scripting vs. GUI applications, the "recording" on actions is nice in theory, but is practically useless in reality for anything but the most simple cases of repetition. Better solution is the model you find in many Linux tools: the actual program that does the work is a pure command line application, and as such scriptable as anything. But since it's scriptable, you can also build a GUI that does the scripting for you and the GUI can be complete separate program.

[Crazed123]

As for scripting vs. GUI applications, the "recording" on actions is nice in theory, but is practically useless in reality for anything but the most simple cases of repetition. Better solution is the model you find in many Linux tools: the actual program that does the work is a pure command line application, and as such scriptable as anything. But since it's scriptable, you can also build a GUI that does the scripting for you and the GUI can be complete separate program.

Well I got ninja'd.

Anyway, I would note that *nix systems actually include a system call "int system(const char *command);" that invokes a one-off shell with the given command (or script!) and returns the return value of the command. GUI programs could indeed build scripts or shell commands up as strings and then just pass them to the operating system for execution.

[mystran]

Anyway, I would note that *nix systems actually include a system call "int system(const char *command);" that invokes a one-off shell with the given command (or script!) and returns the return value of the command. GUI programs could indeed build scripts or shell commands up as strings and then just pass them to the operating system for execution.

Anyway, I would note that ANSI C library actually includes a library procedure system() which in a POSIX system is typically implemented using at least the system calls fork(), execve() and wait(). You can use some other exec* variation, and/or waitpid() instead of wait(), but you get the point.

In any case, system() is not a system call. It's a C library procedure.

[Crazed123]

My bad! I've been working in kernel-land for too long.

[Brendan]

Hi,

All applications should accept the "--hidden" command line argument that starts the application in a special mode where no window is created. This means if you run a script that uses several different applications you don't end up with the screen going insane as applications are started and stopped (but you could omit this argument, so that you can write a script that launches an application and does some stuff in advance before the user takes over).

Got any way to make all applications conform to your standard? It's a good idea, but I don't see how the system can enforce it.

There is no need for the system to enforce this, just like Windows doesn't enforce a "file" menu that contains "open", "save" and "exit" options, and *nix doesn't enforce the "--help" command line argument. It's a convention that would be used where appropriate to make things easier for users. Of course for some things it's not appropriate - for e.g. a tiny utility that ejects the CD might never need to open any window (and might be useful at the end of a script that backs up data to CD), and a utility that does nothing but display a JPG wouldn't make much sense if used with "--hidden".

There should be a compiler for the scripting language, and people should be able to write normal applications and games using the scripting language (and sell the resulting binaries like any other commercial software if they wish).

Eventually, the OS itself should be ported to this scripting language, and then the scripting language should be ported to other architectures.

Your idea runs into some implementation trouble here. The main point of a shell or scripting language is to treat full-fledged programs (in binary or in another interpreted language) as simple subroutines of the script.

However, an operating system also has the imperative to protect programs from one another. So you can't have a scripting language capable of declaring a pointer to a program's internal memory, because the operating system runs the script interpreter (or compiled script) in a separate protection domain (usually a separate address space) from the invoked program. And that requirement would cripple the language for writing the operating system itself.

Nothing says a language needs to support or use pointers, and there are enough examples of languages that don't. Pointers (and inline assembly) are required for operating system kernels and boot code (unless you hide it in a HAL I guess), but for a micro-kernel the kernel and boot code is only a small part of the entire OS, and everything else can be written in a language that doesn't support/use pointers.

Finally, how would such a scripting language approach concurrency?

I'm not sure I understand the question (seems fairly simple to implement to me). Can you imagine any sort of problem with supporting multi-threading in an interpretted language?


Cheers,

Brendan