Second Part

0:09:07 McLuhan’s notion was very important to me; as a student I printed up little business cards saying 'Ben Shneiderman general eclectic - Progress is not our most important product' which was a play on the General Electric slogan of the time; I came to accept the idea that I may not be the best at any specialism but I might be the best generalist; though in that competitive atmosphere that I was in to succeed you had to be the best of something, a harder thing to do was to be the best generalist; professionally it is not an ideal choice although it has been useful and satisfying in my life; those who chose a specialist path and follow it have an easier career; my own approach meant I had to go from interest to interest, whether photography or physics, computing and applications of computing into many different fields; I have collaborated with political scientists, English scholars, chemists, and biologists, and I find these enriching, and have given me a set of multiple perspectives that I find useful for almost any problem; I can make conversation in a lot of different circumstances; I came to feel that after six months of working on a problem with political scientists about Supreme Court and District Court citations over a hundred and fifty years, I was knowledgeable enough to make contributions in the field; the political scientists would publish their findings while I would publish the computer aspects with my students; so the capacity to look at a problem both as a specialist in one problem but generalizing to another problem is probably my best skill; five or six years ago I began to work with molecular biologists about gene expression data analysis; I found it fascinating but a bigger mountain to climb than anything I had done before; after three and a half years of working on it, producing results, I did not feel I had achieved that proficiency to be able to discuss and carry forward in that field; I had a weaker high school biology training and the gap between what I knew and what I needed to know was quite large; I had thought our work should get an acknowledgement but the ethic in biology is that all names go on papers, so there are papers that I am author of but I could not explain them to you adequately; it is a challenge to me, because the conflict between disciplines and their ways of working is unsettling; I felt for a while that I should put those biology papers in a separate section of my resume because I didn't feel I could claim the content of those articles; it is certainly different from computing where individual authors or small groups would be typical; however, the eclectic style of working suited me and I enjoyed the sequential relationships with professionals in different fields to solve their problems and stimulate our work in terms of new problems; the current example of electronic health records is yet that problem again; I think working in medical computing is a noble application of computing which is in harmony with this tikkun olam notion of mending the world; we also broaden computer science not only by its application but by bringing new problems that had not been attended to before; my colleagues think that when I work with English professors that I am doing them a favour; they rather disparage true interdisciplinary work but these English scholars are brilliant and the challenges they bring to us are a novel and important expansions of computer science; by having this sequence of collaborations and working in interdisciplinary teams I do have to establish the ground rules with the scholars that we are not there as their programmers, but are partners; we need to publish in the computing literature and my students need to get PhDs in computer science not merely programming for them; this they come to understand; we have been successful as a group as we have kept explicit several of those principles; the breadth of my reading is great, not just in computer science, but in the broader areas that I have worked in

9:04:04 On graduation from City College with a bachelor's degree in maths and physics I was looking to go to graduate school in computer science; by that time I was more knowledgeable about which universities were good or bad; the favourite place was Carnegie Mellon University to work with Allen Newell and Herb Simon who were then working on the early stages of artificial intelligence; I went to Pittsburgh, paid my down payment, excited that there was a teletype in the basement of Mudge Hall near my dormitory; the Head of Department was Alan Perlis, a famous computer scientist; the thrill of going to work for Newell was very much on my mind, but it was the time of the Vietnam War and the US draft board had said that my going to graduate school was not an acceptable deferred occupation; Carnegie had given me a fellowship and they then gave me an instructorship so that I would be working; I returned to my local draft board and they still said it was unacceptable; this was problematic and I wound up going to teach at a two-year college on Long Island; I was invited to be a full-time instructor there in the Department of Data Processing; Harold Highland was the department Chair and he took a liking to me; he had a son my age who was in similar circumstances so understood he was helping me; my draft board took this to be my national service; I taught fifteen hours of classes a week on Fortran and data processing equipment in a vocational environment; you teach people a very professional skill and they get a job doing that; I taught there for three years; I worked hard and that is how I came to be at the State University of New York at Stony Brook, also on Long Island near Brookhaven Labs; it was a new university and I came to take classes there; computer science was now emerging as a discipline, there was a department but as yet, no degree; I was also doing applied mathematics; I became the first PhD in computer science at Stony Brook in 1973; another colleague of mine there, John Hennessy, is now the President of Stanford University; we had a good group there among whom was Isaac Nassi; the university, being new, was not a very supportive place; my memorable moment outside my office - (I had come as a graduate student but had become an instructor) - there was an open steam vent which they were working on, and one night a student fell into it and died; it was that kind of lack of care which typified the place; however I had a couple of excellent instructors, Herbert Gelernter among them, an early pioneer in artificial intelligence, geometry theorem proving; his classes were magnificent; remember a conflict I had with him when I had got excited about doing some of the theorems; I offered to write a theorem proving program, which was pretty ambitious, if he would excuse me from the mid-term exam; he refused and really annoyed me, though I think he may have been right to force me to do the basics; my own professor there was Jack Heller who had worked on databases and mathematical theory of same; it was there also that I learnt the practical connection because he had been working on museum databases, and we worked on the Museum of Modern Art’s fine arts catalogue; my dissertation was a theory of these richer data structures called 'The Graph Theoretic Model of Data Structures' (editor: correct title is Data Structures: Description, Manipulation and Evaluation); it had the application from the Museum and also the generalization of the mathematical theory of graphs; I was able to apply graph theoretic strategies to making efficient models of how to implement these things; I had begun the dissertation writing process in the way that I now work with my students; the traditional approach would be to do your work, then write your dissertation, and then publish your papers from that; I was much more into doing the work, publishing four papers, then writing it as an integrated document to make the dissertation; the advantages there was in harmony with my social approach, to get your work out and let people read it, criticize you and find out what is wrong with it in time for you to make changes and adjustment to it; it also pushed me towards having the work done so it could have an impact on the Museum of Modern Art, and other museum projects that Heller was working on; the other point is that real work should be applied; I don't subscribe to the strong separation between basic research and applied research; I think you do your best basic research if you have a real problem; I follow this with my students, even undergraduates, that you must have somebody outside the classroom for whom you are building this; that has been a very productive approach for all students; I see so many times papers that are done in an abstract way; it is true that maybe twenty years from now somebody will find an application, but I don't have time to waste on abstract problems

18:53:24 I met Ted Codd - very British - I was a very early player in that space of relational data model; he was a wonderful inspiration; his February 1970 paper certainly was one of those important moments where he not only presented an idea but also its ramifications; I was at that time working on those problems and visited him at IBM; one of the papers written for my PhD was published in 1973 called 'Optimal Database Reorganization Points'; one of the issues at that time was when you added or deleted items from a database it would become less efficient, and each successive query would take longer; the desire is to reorganize often but that is costly, so you need an optimum strategy; I had a very lovely solution for that, a fairly mathematical piece, which stimulated thirty-five papers in the next few years on this topic, and opened up a new field; as a graduate student that is the kind of thing you want to see happen and that was one of the components of the dissertation; as a graduate student I had been a volunteer for the ACM at a professional event, and hosted the British software engineer, Michael Jackson; he came to speak in New York and was proposing these new notions of structured programming; rather than the 'go to' of Fortran it had a set of structures of 'if then else, do'; Jackson presented this idea and it seemed a natural jump; if you don't have 'go to' then the flow charts that were popular with arrows connecting to boxes were also not valuable; you had instead recursive structures of 'if then else', then looping structures and sequential execution; there was a nice mathematical proof that those three structures provided all the power that you needed in programming; I began to draw boxes nested within boxes and developed a set of box shapes that would represent this; it was a fifteen minute invention while listening to his talk; I returned to Stony Brook and showed my colleague Isaac Nassi as he was working directly on that for his dissertation; he got excited as he knew the justifications and why, and together we wrote this up and he helped expand the idea; we called them structured flow charts - known as Nassi-Shneiderman diagrams, NSD - Nassi prevailed on me to make an alphabetical order to authorship; we submitted this for publication in the 'Communications of the ACM' and it was returned in a few weeks rejected; the rejection letter said the authors should collect all copies of the paper and burn them; quite a shock for a graduate student to get something like this, made me think I might be on to something; others thought it was clever, innovative, useful, so I had sent it to a few people; one was a senior colleague whom I respected; he didn't respond, but a few months later another colleague sent me tear sheets from a popular science magazine in which he had published this idea under his name; that was really troubling as a graduate student; people said I should file a complaint to the professional society; I just left it and for many years it troubled me as many of the books mentioned his name; eventually this got sorted out and it’s widely known as Nassi-Shneiderman diagrams; we then published this article in a non-refereed newsletter of one of the professional societies for programming languages and there are thousands of citations to this paper, hundreds of software packages, patents that follow this, there is an international standard for doing these things; again it was one of those wonderful fifteen minute innovations that we did very little to promote but travelled very well; it is less used now as are most of these diagramming techniques, though it is still widely used in Germany; it is a great episode about the paths of creativity and how certain ideas travel well; maybe the first hundred papers were variations on the theme; it was a simple idea that people could understand and opened up the door to many possibilities; I finished my degree in 1973 and took a position at Indiana University as Assistant Professor; I had recently married and went off to do my service to the mid-west of the US; at that point I was quite strongly motivated to repay my debt to the state universities where I had studied, and Indiana University was a state university; it is a small university town at Bloomington with a small campus; there were five of us junior professors hired the same year, all with families and all Jewish; we became close and all had children; it was a nice experience for three years, and my daughter, Sara, was born in Indiana in 1975; after that, the desire to get back to the East Coast and its neuroses was attractive; I would have liked to have been closer to New York and to family but my wife wanted to be further away; the compromise was to come to College Park at the University of Maryland; there was a particular attraction there because there was a senior faculty member, Ed Sibley, who was head of a small department called Information Systems Management which dealt with the database issues for which I was making my reputation; Sibley was interested in my work on experimental studies of programmers initially; I had learned my trade as a psychologist at Indiana University which had a first rate psychology department with B.F. Skinner; one of the young faculty members there, Richard Mayer, taught me psychological experimentation and I taught him about computing and together we did some good work; he has gone on to a distinguished career in the University of California, Santa Barbara; that was a good partnership and I was gaining some reputation with early studies about programmers; when I was at City College I described working on Fortran and my friend Charlie Kreitzberg and I wrote a little guidebook on how to do good programming in Fortran; we eventually wrote this up in a book 'The Elements of Fortran Style' in about 1971; it had a moderate success and inspired a whole series of other books, but we certainly started that trend; it was a series of rules about how to write good programs, the choice of variable names, modular design, indentation, commenting, a variety of stylistic issues; those were really conjectures based on our experience; I became imbued with the notion of conducting empirical tests where I would make a good and bad version and give them to someone to debug or interpret and I would measure their performance; I was able to alter the independent variable and measure the dependent variable, and that is the way these experiments began; when I came to Maryland this Information Systems Management Department was in the Behavioral and Social Science School not in computing; that was a very interesting opportunity; it was close to the Psychology Department so I was attracted to that; Ed Sibley had been a collaborator and had been helpful to me; early on he had engaged me in doing short courses around the country and abroad on database technologies, before I got my PhD; we had a very good group at Maryland which did well; however it was a small group and eventually lost the political battles; that department was dissolved and most of the faculty left; two of us went to Computer Science so I went to my natural home; I was not especially welcome there as I was seen as a little different; I was into these psychological studies and not especially celebrated; when you bridge into interdisciplinary and newer areas you are going to have some trouble with some fraction of the established community; I had got tenure based on traditional computer science and had demonstrated my mathematical skills

34:37:19 I enjoy lecturing and teaching; early on when it was difficult to get funding for research on the psychological experiments of programmers, I would put my students to work and would make their semester projects to be an empirical study; I was most proud to be able to publish in professional journals with my undergraduates; colleagues disparage that by saying that if the work is so simple that an undergraduate can do it, it can't be very good; but I had done the mathematics, taught mathematical analysis, I can do that stuff, and I moved on to these newer forms which were fun for me; when Sara was in third or fourth grade computing was all the rage at the time; her teacher came and asked me to recommend a textbook to enable them to teach the children the language Basic programming; I was appalled to find that the books were either trivializations or if they taught you programming they depended on high school mathematics; I quickly wrote them about seventy-five pages of programming introduction where the examples were all graphical which kids could understand; it worked well for her class; the DC school system asked me to polish it and they printed 3,000 copies for summer computer camps; we did them for four different computer systems and Sara worked through the problems for one of the versions; I used to help my children with their science projects; I remember one of them was using the sound generator on a computer to generate sound at different pitches and volumes, and to see what was the difference that people could determine, in amplitude or frequency

38:50:18 By 1980 I had taken the different work I had been doing on the experimental studies of programmers and wrote a book called 'Software Psychology', a kind of marriage of disciplines; it was taken up by both computer science book of the month clubs and became a hot topic; by 1982 a group of us were meeting in Washington; I had been the organiser of the Software Psychology Society whose claim to fame was “no members, no officers, no dues”; we had a mailing list of 600 and forty to sixty showed up each month at George Washington University and I organized the speakers; for twenty years this was a very influential group that had both local and national impact; we decided to run a conference in 1982 where we hoped to bring together two to three hundred people; it was held at the National Bureau of Standards and Technology and we drew over 900 people; people were suddenly interested in the topic; the personal computer had appeared though it was still in the days before the Apple Macintosh, but these ideas were beginning to grow; the following year the ACM Special Interest Group on Computer Human Interaction was formed and they took over the conference, and it has continued to be the main source drawing as many as 3,000 every year; that bridge of disciplines was the natural spot for me to be in and I am pleased that it had succeeded; most computer science courses (departments) have one called Human-Computer Interaction; there is still some resistance in computer departments but it is definitely a well-accepted field; by 1986 I published the first edition of 'Designing the User Interface'; in the fourth edition I was joined as co-author with Catherine Plaisant who had been my research collaborator since 1987, and we have just released the fifth edition this Spring; that has been the main textbook in the field and has helped define it; my satisfaction has been to see that go; the early edition of the book focussed on a small group of users like programmers and medical specialists; by the fifth edition we are talking about four billion cell phone users, Wikipedia and YouTube - the emergence of these platforms of user-generated content is a remarkable transformation for which I am proud of my role; in some areas it is seen as new stuff and not quite science, but this softer science is the way that many things will go; I have written on the history of physics, chemistry, controlled laboratory science of the natural world which will continue; however, I suggest there is a need for a new way of thinking which I call Science 2.0, published in 'Science'; this suggests that the content and the methods need to be changed as well; my eclectic background gave me the capacity to see this perspective; not everyone accepts it but I think the notion is in the right direction; it shifts the methods from control of repeatable laboratory conditions to case studies and interventions in the way that you can do on the Web and have ten thousand data points in two hours which give you remarkable capabilities; shows the trajectory from traditional computer science to these newer bridging sciences of psychological applications and a larger vision about the transformation of science, always with the intention of making the world a better place; as a profession it is a difficult trajectory to do because I'll never be accepted as a leader in medical computing, but I feel I have made my contribution; by going from one discipline to another you start out again as a graduate student which I am happy to do; McLuhan taught me to be a dilettante and of being an amateur as a virtue, and starting on something new is satisfying as part of my everyday work; another contribution that I am proud of is hypertext; we worked on this in the mid eighties and our contribution was the link; when you click on highlighted words and go somewhere, that was our work; many people had commented on the link which is usually traced to Vannevar Bush's article in 1945 which talked about Memex; his notion of the link was that you type in a code number for a document and your microfilm devise with spin to get you there; we began to develop these electronic encyclopaedias and there was an 'ah ha' moment when we had a videodisc player for the images and a plain green screen for the text; we had captions and descriptions and then a numbered list of where you wanted to go next; in one screen there was a very short caption but a numbered list of Polish poets; the text in the caption and the list of names was almost identical and I asked Dan Ostroff, the programmer at the time, how we could replace the caption but then highlight the names and click on them; that was done and we did a number of studies comparing different forms of highlighting; we developed light blue underscored,  Tim Berners-Lee saw that and in his 1989 manifesto for the Web he cited our work; it was a small contribution but had a broad impact and gave us great satisfaction; there is controversy about whether we can take pride in this claim - others were certainly working on this - but we made this particular way of doing it, provided an authoring tool and a rich environment to support all that, and that became the inspiration for Tim Berners-Lee; we had published this under the term 'embedded menus'; Tim was clever enough to call them hot links

49:51:22 Tim Berners-Lee deserves credit for inventing the Web; we were working on hypertext things that ran on a single computer and his mind-blowing idea that you could have such a link and it would jump to another computer across a network to retrieve it was wonderful; I met him before that at the Paris hypertext conference; we were building systems called Hyperties which had a commercial version as well, so we were an active player; we built the world’s first hypertext scientific journal - the July '87 issue of the CACM; we did the worlds first electronic book called 'Hypertext Hands On' which was in a paper form but it was the disc that mattered; it is the first book in the Library of Congress that has electronic media as part of it; we definitely made those things happen at scale and in marketable ways to reach thousands of people, including Tim Berners-Lee; I am on his scientific council for the web science research initiatives so see him a couple of times a year

51:17:14 On the future, I have already hinted at some of this in the 2.0 argument but the expansion of social media, of user generated content, the empowerment of individuals, and the restructuring of social and political norms are beginning to happen; as you take Facebook and YouTube and apply them to national priorities like energy, health, education, sustainability, environmental protection, you get a transformative effect; I think that is what is really dramatic; between the fourth and fifth edition of 'Designing User Interface', this has just flourished in a way I would never have predicted; YouTube which wasn't around five years ago is now number three on the Web after Yahoo and Google is a startling indicator; I think those things are only just beginning to unwrap in their implications to society; the new forms of media with the dissolution of newspapers in the US and the profound restructuring of societal norms is yet to be fully appreciated; when journalists ask me what is the next killer app I have a straight answer which is trust, empathy, responsibility and privacy; it is those who understand how to generate trust, support empathy, how to make responsibility for failure and success more clear, and protect privacy, also how to raise the level of motivation so that more people participate and contribute, what's the role of egoism, altruism, communalism, and how do we make these social structures in new ways; that is more or less the way I see the world; revising social structures, political structures, economic structures is a fascinating potent transformation; how medical care with change and people become more responsible for their own medical records and treatment; working on innovative social structures is as interesting to me as working on innovative algorithms or devices; I have had the satisfaction of seeing our more recent work on visualization become a commercial success with Spotfire; the idea which we published in a 1994 paper is still one of the most widely cited in the field with Christopher Ahlberg as a visiting student; he formed the company in 1997 which grew to two hundred people and was bought in 2007; it is nice to see a success story like that, and I think people appreciate our Lab, not only for lots of papers and good students, but we have had success stories on the commercial side too; Spotfire and Treemaps have a dozen or more widely used open source versions as well as others; the intellectual challenge is to know which idea will travel well; I am always trying new ones, putting out seeds, and some go well and others struggle; my efforts working on creativity support tools has produced a moderate result but has not become a large field; other ideas like the link on the Web or the notion of human-computer interaction, some are small focussed, others broader restructuring of disciplines; people think I do good demos, but finding ideas and developing them in a way that has an impact and then communicating them effectively is what our job is; sometimes it works well and sometimes I struggle, but still want to get ideas out there