Wednesday, January 22, 2014

Google Glass is the API to control humans (and it's not as bad as it sounds)

With Google's acquisition of Nest, it occurs to me that Google Glass may have more far-reaching ambitions for human society than most of us can imagine right now.  And it has less to do with serving the needs of the person wearing the Glass, than controlling that person to do things that the Glass is telling the person to do.  So cynics can rest comfortable with their critiques of whether or not Glass is useful -- it may not be, to them.  But that's okay.  In other words, Glass may be the beginning of the API -- application programming interface -- to control humans.

Hold on, wait, it's not as bad as it sounds!  Really!  What I mean is augmented intelligence, not mindless drone control.  Okay, maybe it's a little creepy.

A few years ago I was lucky enough to be in Cambridge, Mass. to meet Sep Kamvar, a former Googler and now a Professor at the MIT Media Lab.  Amongst other fascinating things he described (e.g. he felt iGoogle had made some missteps with respect to the curation of its early developer community members), he posited the possibility of programming languages to interact with real-world, meat-space problems, not just objects and methods in digital space.

For example (my example, not his) just imagine if Andy Samberg and Justin Timberlake could write Javascript to do the following, not just in virtual space, but in real world:

box.cut(hole)
box.put(junk)
box.open

APIs to interface with the real world could be thought of as, maybe, robotics.  But think of where Amazon has gone with Mechanical Turk -- Human Intelligence Tasks where you can describe a task for a human to perform, and interface with a computer system.  As Amazon describes this, it is Artificial Artificial Intelligence.  I like a new platform called TaskRabbit, where you can basically find help to do stuff in real life -- household errands, skilled tasks, etc.  TaskRabbit can basically match up extra labor supply with sporadic demand.

So what if the end-game for Google Glass isn't just an interface for the wearer to be the user?  What if, instead, it's the best way to seamless strap a control mechanism onto another human?

Creepy right?  Well, consider the use cases in health care.  What if you have someone who's a relatively skilled and proficient with their hands, but only a novice with respect to healthcare.  What if you could assign tasks and walk that person through how to do anything from ask the appropriate questions and look for the appropriate signs (physical manifestations) to diagnose malaria?  Or to administer the right treatment?  Or to perform a surgical or invasive procedure?  What would it mean to global society if suddenly we didn't have to take a whole decade or more to train a doctor, but could mobilize masses of lower-skilled workers and put Google Glass on them to enhance their skill set and knowledge set, augmented by an instruction set designed by the world's best doctors and engineers?

In the same way that you can imagine Cyrano de Bergerac whispering and coaching Christian beneath Roxane's balcony as to what to say, what if someone -- or something -- on the other side of Google Glass could enhance the wearer's abilities beyond what was ordinarily humanly possible in a world where they'd otherwise need to take the time in school to learn and train on whatever it is they were supposed to do?

If you've ever watched the TV show, Chuck, or even better, a 1969 Disney live-action classic called "The Computer Who Wore Tennis Shoes", you'll recognize the similarities here.  In those two examples, a human being was magically gifted with ungodly amounts of knowledge that they could recall instantly.  In the TV show Chuck, Zach Levi's titular character was not only able to "flash" on knowledge, but eventually do Matrix-style instant-learning of physical and real-world skills -- say, martial arts fighting styles, flips, fencing skills or how to speak a new language.

And, just so we don't forget the implications for healthcare, check out 7:40 in this compilation of clips from Chuck, where Chuck "flashes" and instantly learns surgical skills to be able to safely extract a bullet from a spy-friend's leg.

Oh, and for those of you who somehow missed the last decade, here's the scene from the Matrix where Keanu Reeve's character, Neo, learns jiujitsu in an instant.  By the way, everyone remember's Neo's line "I know Kung Fu" from that scene.  The real important line slips away almost undetected in that scene, when Tank says: "Ten hours straight... he's a machine."  Neo has been boosted -- programmed -- to become super-human -- machine-like.

So I know we'll have our share of thoughts on how great Google Glass will be as a heads-up display for the wearer of Glass to, say, look up information, or get contextual alerts.  But I think the gang at Google X maybe has greater ambitions -- and that is, to have a mechanism to see what the user is seeing (the front-facing camera), where their attention might be fixed (the wink camera), what they're hearing (the speech-processing microphone) and finally a way to "control" or transmit instructions to the user (the prism display and the earphone/bone conduction audio setup).  Even in its most basic form, it can be a brilliant, anywhere-capable teleprompter, which anyone from the President of the United States to Anchorman Ron Burgundy would probably be able to acknowledge wields tremendous power.

What if, for example, you could pack at least one Google Glass on every airplane?  That in the rare case that an emergency arose, you didn't hear flight attendants asking just if there were medical professionals on the plane, but (especially if there were WiFi on the flight) donning a set of Google Glass and getting walked through -- either by A.I. or a human supervisor or both -- the interactions and actions necessary to safely see the patient to safety?

What if you could keep a Google Glass unit everywhere you had reason to staff people who knew CPR, or to keep a defibrillator around?  Sure, maybe people might know CPR or what to do, but what if you could instantly convert any able-bodied person into a CPR-capable helper?

Or what if you could give every school nurse a Google Glass unit?  They're already a healthcare professional -- now let the school nurse interface with a team of pediatricians ready to help them handle any more advanced issue, even if only to help triage it?  My daughter was in a kindergarten class with another girl who had Type 1 Diabetes.  Her parents were at wits end, having to constantly come in to accompany her on field trips, or in visit with the nurse and help guide the care of her insulin-dependent diabetes.  What if Glass had enabled them to join and watch over her -- or a pediatrician -- remotely?  To guide an able-bodied helper, or the school nurse, or even their daughter directly, in the care and management of her Type 1 Diabetes glucose level checking and insulin injection?  Of course, to realize this vision we'd need a lot more sensors around.  One could argue for the need for wireless-capable "smart" sensors, like a Bluetooth glucometer -- but really for the first version, if you had Glass, all you'd need is the observer/supervisor to be able to see the glucometer reading from the Google Glass video feed.  Indeed if Google Glass's optical character recognition grew to be as good as its speech recognition, it might be able to parse that data and realize that, in the field of view, there was some alphanumeric data to process and hand off to other pieces of software.

Sure, the dream is to someday have a holographic, artificially intelligent doctor, like The Doctor on Star Trek: Voyager.  But what if, as a stepping stone to artificial intelligence, we might do something that might be described as "augmented intelligence"?

I for one, welcome our new google overlords.  The MIT Technology review had theorized that innovation and technology destroys jobs.  In a June 2013 article by David Rotman, he points out that Professor Erik Brynjolfsson and collaborator Andrew McAfee note that industrial robotics and other advances in productivity (self-driving cars anyone?) will create reductions in demand for many types of human workers.  But this is predicated upon the notion that somehow that innovative technology might not augment the capacity for humans to upgrade their mojo, so to speak -- that is, to take these obsolete workers and still render them useful as actors in the physical world.

It's downright Shakespearian if you really take a step back and think about this eventuality for Google Glass -- as the beginning of an API to control human beings.

As You Like It
Act II Scene VII

All the world's a stage,
And all the men and women merely players:
They have their exits and their entrances;
And one man in his time plays many parts,
His acts being seven ages. At first, the infant,
Mewling and puking in the nurse's arms.
And then the whining school-boy, with his satchel
And shining morning face, creeping like snail
Unwillingly to school. And then the lover,
Sighing like furnace, with a woeful ballad
Made to his mistress' eyebrow. Then a soldier,
Full of strange oaths and bearded like the pard,
Jealous in honour, sudden and quick in quarrel,
Seeking the bubble reputation
Even in the cannon's mouth. And then the justice,
In fair round belly with good capon lined,
With eyes severe and beard of formal cut,
Full of wise saws and modern instances;
And so he plays his part. The sixth age shifts
Into the lean and slipper'd pantaloon,
With spectacles on nose and pouch on side,
His youthful hose, well saved, a world too wide
For his shrunk shank; and his big manly voice,
Turning again toward childish treble, pipes
And whistles in his sound. Last scene of all,
That ends this strange eventful history,
Is second childishness and mere oblivion,
Sans teeth, sans eyes, sans taste, sans everything.

Monday, January 13, 2014

Ok Glass: scribe for me

Human interaction is probably going to change because of Google Glass and/or derivative technology.  I think it's easy to overlook the speech recognition baked into Google Glass; it's really quite stunning (and to be fair I've been impressed by Apple's Siri as well, along with the Mac OS X continuous speech recognition capabilities for dictation). Terms like "pheochromocytoma" (a type of tumor that precipitously secretes adrenaline into your bloodstream) are handled with aplomb.  We think of this as a nice-to-have, but really this is a critical aspect for clinical medicine -- how frail are we, as human doctors, in recognizing everything the patient has said?  If the patient mutters under their breath that they've had ankle pain when they're on a fluoroquinolone, does the physician brain even recognize they said that and tie it into the issue at hand as a possible side effect?

Scribing, it turns out, is an important function.   Katie Hafner writes about it in yesterday's New York Times, in an article titled "A Busy Doctor's Right Hand, Ever Ready to Type".   An ENT surgeon and close friend of mine has often yearned for some sort of system that could automatically scribe your entire interaction with a patient -- or the very least the highlights.  When you have a scope jammed up someone's nose it's not really a great time to be scribbling something.  Charting, it turns out, is a pain.

Writing physician notes -- or typing them while your patient is talking -- can be a good process in and of itself though.  For the non-doctors out there, it's important to understand the difference between the type of medical chart note written by a medical student or trainee, and one written by a full-fledged attending doc (or more senior resident).

When medical students first start out, they learn a structure, to be complete and avoid missing something. This can be agonizing to listen to because they're working their way through a very standard structure, which almost literally goes head to toe. As a result, their medical documentation also is very complete to the point of containing a lot of stuff that really doesn't weigh into our diagnostic decision making -- like a census surveyor trying to catalog every detail of your illness.

By comparison, when you listen to an attending physician take a history from a patient (i.e. interviewing them), they're problem-focused. In addition to already having a good sense of the patient's history, they'll zoom in on hypotheses going through their head -- we call this differential diagnosis -- and ask questions to refine those hypotheses. One example of this process is "Guess the Dictator/Sit-com character: http://www.smalltime.com/Dictator, although that's more of a decision tree than a probabilistic Bayesian process.

Anyhow, the point is that the note the attending scribbles down appears brief -- one of my favorites from a legendary cardiologist at NY Presbyterian - Cornell read, simply, "LASIX!". But it reflects a thought process, a sort of filtration and re-prioritization of what matters.

Google Glass, therefore, could be a very interesting scribe. But a computer could be a scribe, too, right? Well, this is where the display comes in. Like a human scribe in the exam room, who knows just enough to be clever, Glass could point out simple things -- like, say, inconsistent stories in a pain medication-seeking patient. Or search against common mispronounciations of drugs to speed up the game of "I take this pill, I think it's called, umm...." These sorts of things are handy for a doc to be able to glance at as they come up.

In a more advanced form, you'd tie this sort of natural language processing to start parsing out and tokenizing the text. The reason for this is that you could start applying Clinical Decision Support rules. Clinical Decision Support is what separates quality measures reporting from action and improving outcomes. As Dr. Jacob Reider at HHS/ONC might put it, it's like the difference between knowing how to give kids a C- grade vs. giving teachers a way to help those kids improve their grades.

So in addition to scribing, Glass should eventually be a nice mechanism to point out "wow they just described Lyme disease... you want to ask this other question to clinch it?" I think of systems like Dxplain for that, to improve diagnostic certainty. Other systems have aimed at diagnosis as well, and certainly there's a fascination with artificially-intelligent diagnosticians. IBM Watson is a trendy example of this.

But it turns out that another arena of Clinical Decision Support is just common stuff that docs forget. Like remembering to prescribe a medication called an ACE-inhibitor in a diabetic with early kidney damage called microalbuminuria. I once led a chart review study of patient records looking at the cause of gaps in care, like this example, to see whether it was the doctor who had failed to prescribe the drug, or if the patient had failed to fill the prescription. While we think of patient non-adherence as a huge issue, it turns out that if the doctor forgets or neglects to write the prescription, there's no chance for the patient to be non-adherent. And, as the study turned out, doctors just not remembering to refill medications was all too common -- the medication was nowhere to be found in the chart, rather than the medication written in the chart but the patient not picking up the prescription.

This begets a whole user interface and usability issue -- what's the best way to offer a tip or correction to a doc? Too many and they get alert fatigue. Too few and they'll just keep making the mistakes they're destined to make. Glass offers a great way to slip a message in unobtrusively in the heads-up display, to the point of even remembering just to ask another question. Likewise, Glass could listen for the appropriate action as the result of the prompt/alert; e.g. once you said "Mrs. [Patient], I'd like to start you on a drug called lisinopril" Glass would recognize that you had fulfilled the clinical decision support prompt and dismiss it for you.

So speech recognitions is, I believe, not only a critical part of Glass, but an incredible enabler of a potentially much more fluid experience between doctors and patients -- and, if driven by Clinical Decision Support, a whole lot better for quality as well.

(Next up: medical student James X. Wang. Fear not: we'll get to the neurologist a little later!)

Sunday, January 5, 2014

OK, Glass: Start a Timer

When Dr. Wei offered to let me try Google Glass I was unquestionably excited. It’s not every day that you get to try out a piece of hardware billed as the first ever mass-market wearable computer. But, as someone fascinated with and regularly engaged in using new technology, I’d read dozens of reviews arguing that Google Glass is not the prototype for future wearable computers. I won’t rehash those arguments but I do want to make some general points about wearable computers and their use in scientific and medical contexts.

I am a 3rd year MD-PhD student, meaning I spend a lot of my time wearing gloves, mixing stuff in test tubes, setting timers, and waiting for “science” to happen. For me, the natural first destination to try out Glass was therefore in the lab. “OK, Glass. Start a 5-minute timer,” I said aloud, ignoring sidelong glances from curious colleagues. Glass stared back at me blankly and did nothing. It took about 10 minutes of fiddling (and some input from Benji) to figure out how to do anything with Glass.

Once I got Glass working, I found that it had limited utility in the lab. For instance, sometimes I save time by pulling out my iPhone to take pictures of labeled test tubes. While taking photos with Glass is trivial, retrieving them and zooming in far enough to see the labels is difficult. Looking up information or doing calculation was also problematic. Even when Glass understands you and searches Google for the right thing, it insists on reading search results out loud to you—certainly not the most efficient way to retrieve information!

I was most excited to read documents on Glass. For instance, I wanted to see if I could read an experimental protocol on Glass as I was doing the experiment. In practice there was no easy way to load documents onto Glass—and anyway it is a strain on the eyes to read long-form content off of the Glass screen (the built in NY Times app is almost unusable for this reason).

Leaving aside the user-interface issues that will surely be fixed in the coming months, I want to focus on two choices Google made that will limit how Glass can be used.

Here is how you actually start a timer:

(1) Tap on the side of Google Glass (or look up) to activate the screen and microphone.
(2) Say “OK, Glass. Start a timer.”
(3) Use your finger to scroll through a clunky hour/minute/second wheel to the desired time.
(4) Tap on the side to open the context menu and select “Start Timer.”

Notice that that you have to touch Glass multiple times. In fact, most verbal commands require you to also interact with the device using your fingers. This is a non-starter when working in the lab because you must take off gloves, play with Glass, and then put gloves back on. The clunky user interface can be fixed, but requiring finger navigation makes Glass unusable in any setting where gloves are involved, e.g. in the lab or during surgery. Glass will only be useful in these settings if Google can drastically improve the voice recognition software. This brings us to a general point about wearable computers in a medical and scientific context: the richer the hands-free interface, the greater the utility in the lab or sterile field. Google has a long way to go before the verbal interface alone can drive the device.

Equally important to hands-free interfaces is how Glass enhances the world around you. In short: Glass doesn’t. It provides a small HUD (heads-up-display) in the upper right hand corner of the world that can notify you when, for instance, you have a new email or a phone call. It cannot superimpose content onto the world, the way that some other wearable computer products do. This means that Glass can’t help a construction worker position a beam, or indicate which test tubes a scientist has already added reagents to, or overlay a CT scan onto a patient’s body to guide a surgeon’s scalpel. This lack of world-enhancement was surprising and I sincerely hope that Glass is working on a different version of Glass that can accomplish some of the truly futuristic possibilities that people have been dreaming about for decades.

As a laboratory scientist, future doctor and someone fiercely optimistic about how technology can make science and medicine faster, better, cheaper and more accurate, I was unimpressed by what Glass had to offer but remain excited to see how wearable computers will enhance the laboratory and the clinic.

Ron Gejman
@rongejman

Ok Glass: teach a doctor to intubate.

Happy New Year! This post is about Academic Emergency Medicine & Google Glass, and its potential application in supervising trainees... and then winds up at Atul Gawande's observation about the need & lack of coaching for surgeons and medical proceduralists.

Last week I had the opportunity to sit down with an academic emergency medicine physician from NYC, who commented on the sheer lack of tools to help supervise certain medical procedures being performed by Emergency Medicine residents (physicians in training).

One of the most critical scenarios is intubation -- where docs jam a breathing tube down the throat and you hope that they've hit the right place. Best case, they've landed in the trachea, right before divides into two parts -- the bronchi -- in the lungs. Worst case, they've landed in the esophagus, and instead of delivering oxygen to the lungs and helping the patient live, they're basically providing oxygen to the stomach.

I've seen attendings supervise this, particularly in anesthesia (who are masters of airway management -- this is often under-appreciated by patients). The resident will use their intubation scope to lift the tongue up and out of the way, and then the attending will stand behind them like an umpire behind a catcher at a baseball game. And, like the umpire, if they have a line of sight (the resident ducks their head out of the way briefly), they'll be able to see right as the tip of the endotracheal tube is about to enter the trachea. But it's a pain because in that fleeting moment as the attending takes a look, the resident has to move their head out of the way.

Enter Google Glass. If the camera is close enough to the line of sight of the resident physician doing the intubation, they can get a shot of the ET tube as they try to land it in the trachea. And either via Google Hangouts video-conferencing, or via the MyGlass app that can mirror what's going on in the Glass unit,

The same goes for multiple procedural specialties, from an emergency needle decompression of tension pneumothorax, to a lumbar puncture, to the more mundane insertion of an IV line. As an attending physician, I could be at another clinical site, in a call room, or I could even be at home in my jammies at home, woken up in the middle of the night, to quickly log in and lend an extra pair of eyes to the resident -- or intern -- performing the procedure in the middle of the night.

Or, imagine if there were a supervisor centralized across geographies -- what if Google Helpouts (https://helpouts.google.com) weren't just for consumers, but for professional continuing education as well? What if the nations' best doctors for XYZ procedure were available to watch you perform a procedure? You could be 30 years into your career, and want someone to peek over your shoulder (or through your Google Glass), and give you coaching on your technique.

In a 2011 issue of the New Yorker, Atul Gawande has remarked on the utility -- and rarity -- of personal coaches for surgeons and medical proceduralists like interventional cardiologists performing cardiac caths, or gastroenterologists. Link: http://www.newyorker.com/reporting/2011/10/03/111003fa_fact_gawande?currentPage=all

The only problem with this is that it's hard to find docs that want to supervise other docs, and handle the logistics of getting them into the OR or at the exact moment you happen to be doing the procedure (e.g. in an emergency room, for example). And also, as Dr. Gawande points out, there's the bigger issue of medical culture -- that "we may not be ready to accept -- or pay for -- a cadre of people who identify the flaws in the professionals upon whom we rely, and yet hold in confidence what they see."

So let me ask you out there. As a patient: would you rather have a medical or surgical procedure performed on you as-is today, or with one of the country's experts in that procedure able to "peek over the shoulder" of your doctor using Google Glass + Google Helpouts? And: how much would you be willing to pay for that? What if it was a procedure for your child?

Up next (hopefully): a neurologist's views on his time with Google Glass. Stay tuned!

-Dr. Wei