What’s New in Surgical AI: 6/11 Edition
What’s New in Surgical AI: 6/11 Edition
WWDC 23 and DARPA for healthcare?
Welcome back! If you’re new to ctrl-alt-operate, we do the work of keeping up with AI, so you don’t have to. We’re grounded in our clinical-first context, so you can be a discerning consumer and developer. We’ll help you decide when you’re ready to bring A.I. into the clinic, hospital or O.R.
Today we’ll be going over Apple’s announcements from their worldwide developers conference and how it could impact medicine, and break down a new government program meant to mirror the defense research organization DARPA.
Table of Contents
📰 The News: Apple Enters the AI game…with a Headset
🤿 Deep Dive: ARPA-H for Surgeons interested in AI/ML
🪦 Best of Twitter: RIP
📰The News: Apple Drops VisionOS
Apple had their annual WWDC (worldwide developer conference) this past week, and they launched their fist major product since the iPhone… a $3500 headset known as Vision Pro
Imagine your Mac or iPhone but displayed in your real world. That’s the sell. The product is set to ship in “early 2024”, but developers got a chance to test it out at the conference. The results were overwhelmingly positive. Some of the quick hits included:
Eye and hand tracking were nearly perfect (more on that soon)
Latency was near zero (more on that soon)
The ability to interact with your normal environment + Vision Pro’s environment was key
Check out this wild thread on how Apple used what they describe as neurotechnology to correlate eye tracking movements and finger movements with predictive analytics to be able to pull/load/predict the app you’re going to use before you even start reaching for it.
At a $3500 price tag, it’s clear the market for these will not be the everyman - but that’s what happened with the iPhone at its original release as well. Lets quickly jump into some medical use cases and times our mind was blown, thanks to Apple:
Finger, hand and eye tracking was perfect. Are you kidding me? Apple placed 12 sensors/cameras around the goggles and can pick up subtle movements. This is a game changer for predictive analytics, surgeon movement tracking, and the entire field of surgical data science - if we can get our hands on the development kit.
This might be an early version of a consumer oriented Brain Computer Interface. Taking pupillary movements, likely saccade-like and before even voluntary motor function has been enabled, is clearly moving along the brain-computer-interface timeline. Who know Apple would be the one to productize “the eyes are the window to the brain”?
What will be interesting to see is how the visual information is displayed relative to the real world. This could have enormous implications in the OR - if data could be provided to a surgeon, without being obtrusive to the field of view (but still visible) … they might actually use it. I am hopeful.
I hope the world of inpatient rehabilitation is overhauled with VR applications. The ability to gamify, transport patients to places they want to be, or worlds that don’t exist, could hopefully be an incentive to get up, moving and out of bed.
Interestingly enough, Apple did not mention “AI” at all during the WWDC. Instead they brought up machine learning, a subset of AI, plenty. But the reality is different: everything from face recognition, to background erase, to neurotechnology has a large machine learning backbone powering its intelligence. This is in classic Apple fashion - disguising the highest of technical excellence as simply Apple Magic.
Bottom Line:
The new AR/VR headset by Apple defined an entirely new product category. Despite the hype (and the hefty $3500 price tag), everyone who tried it on almost unanimously said it was fantastic. The use cases in medicine for really good AR/VR are strong. We’ll keep you updated as we learn more.
What do you think? Will you be buying one…For science?
Other News:
Carbon health announced an AI-powered ambient listening service which they’ve rolled out to over 500 providers, which reduces the time taken to write notes by 60%. We recently dipped our toes into this arena as well, and launched dotphrase.ai, a safe easy and fast dictation and documentation service powered by A.I. (Disclaimer - we built it). You can try it out here.
Google released their model playground for Google Cloud. This is a big win for enterprise players, as it means small teams (or single developers) can play around and fine tune Google models on their own proprietary data that’s stored on Google Cloud.
This was a common complaint amongst hospitals who couldn’t let their IT build on OpenAI - now any GCP clients can go ahead and do so quite freely and (importantly) with small teams.
The playground also offers some handy features like making it easier to choose what format you want the output in, and a place to give the model examples of how you want a conversation to go (this concept is called few shot learning which we will get into next week)
🤿Deep Dive: ARPA-H for Surgeons interested in AI/ML
For this week’s deep dive, we’re going to take a closer look at a new federal research agency devoted to exactly the kind of research we talk about every week in the substack, the Advanced Research Projects Agency for Health. This is a new funding mechanism with at least $1B in appropriations (could be as high as $6.5B, or more). We’ll cover why it exists, what it might do, and how you might want to consider interacting with it.
Backstory: The other ARPAs
ARPA-H is the newer, less self-confident sibling in the ARPA family. The oldest and best known member of the family is the Defense Advanced Research Projects Agency (DARPA). DARPA was founded in 1958 by President Eisenhower with an explicit mission of regaining supremacy in military technology through high-risk, high-reward scientific research. With space research largely carried out by NASA (created the following year) and the respective military branches, DARPA’s initial funding of $520M ($5.46 billion in 2023 dollars) was devoted to transformational activites. DARPA’s work led to the creation of the systems that would eventually become the internet (ARPANET, hypertext), global positioning (NavSat), ballistic missile defense, hypersonic flight, and many other systems. In addition, DARPA programs and contractors have access to a significant fraction of the ~$80B classified “black budget” through BRIDGES and other programs. Honestly, the history of DARPA is fascinating and could occupy many volumes of this newsletter, if its tales could ever be openly discussed.
But DARPA isn’t the only ARPA out there. There is an ARPA-Energy, a Homeland Security ARPA, an Intelligence ARPA, and a number of other federally funded R&D centers that are supported by these and other (perhaps, even more secretive) agencies.
Why ARPA-Health?
Given this plethora of existing agencies that often fund biomedical research, and the existence of explicit science and medical funding agencies such as NSF and NIH (disclosure, I’m funded by NIH NIBIB and planning to apply for funding with ARPA-H), why in the world would we need a new ARPA? And isn’t this just a drop in the $245 billion bucket of US public and private biomedical research?
Unless you’ve been sleeping in a Biohazard Safety Level-4 secured facility for the last three years, you’ve probably noticed that a sweeping global pandemic has catalyzed efforts to create working health care projects from basic research.
This time, we got lucky: moderna’s mRNA vaccine technology needed for COVID-19 vaccines was partially funded by DARPA (P3, ADEPT) in the mid-2010s, waiting to be used.
Most of us who work in the peer-reviewed, grants-funded game know that the system is fundamentally broken. It took me five years to be ready to apply for an NIH K23, three months of almost 100% time to write the application, a further eight months to receive notice of funding, and it’ll take me four years for my project. That’s 10 years. And I got lucky! I was successful on my first NIH application, an extremely rare feat.
In addition, the framework of the NIH avoids proposals that have high probability of fundamental technical challenges, are novel and thus likely to fail. But if they succeed…
What is ARPA-H and how does it differ from the rest of the NIH?
A new agency under the Department of Health and Human Services, requesting $6.5B (~ 14% of the NIH budget), ARPA-H has received $2.5B in appropriations with the first $1B available at the end of FY2024. The first of three ARPA-H offices are located in Washington DC, and by statute, are not affiliated with the NIH campus. The director of ARPA-H reports directly to the secretary of the HHS, not the NIH director. However as a “start-up” agency, ARPA-H will utilize existing NIH infrastructure to some extent.
Unlike the NIH, which typically issues grants, the program managers will likely issue contracts and transactional agreements that have specific milestones and deliverables, with greater accountability and importance of the interaction between the program manager and the contracted research performers. The explicit goal at the end of the program is to graduate contracts to partner companies and organizations to increase scale and implementation.
What will ARPA-H do?
Simply put, ARPA-H occupies the gaps in the existing federally-funded and corporate biomedical research landscape. Ideal ARPA-H projects might be: high risk, eexpensive, require complex coordination among multiple parties, have an applied focus rather than a basic research focus, and have a broad scope that no company can realize full economic benefit. ARPA-H projects are designed to have an explicit exit, “transitioning” these projects into implementations.
The first four focus areas (there might be up to eight, total) are led by autonomous program managers to address challenges that are not easily solvable through existing research and have measurable outcomes:
ARPA-H has adopted a version of the “Heilmeier Catechism,” a set of questions that DARPA uses, to assess program proposals for ARPA-H investment. The questions include … “How is it done today? What are the limitations of present approaches?” and “What is new about your approach? Why do you think you can be successful at this time?” New questions specific to ARPA-H include, “To ensure equitable access for all people, how will cost, accessibility, and user experience be addressed?” and “How might this program be misperceived or misused (and how can we prevent that from happening)?”
The individual program managers and director have tremendous autonomy
There’s certainly a lot of white-space for this team to utilize. As we learn more about who joins the agency, we might get a better sense of how the individual programs will function. The current director, Dr. Renee Wegrzyn, is an applied biologist with extensive experience in both industry and biosecurity at DARPA. Most recently, she was the VP of business development at the controversial synthetic biology firm Gingko Bioworks DNA 0.00%↑. The current deputy director, Dr. Susan Monarez, also has extensive experience in national biosecurity initiatives as well as international cooperative agreements. Her prior focus on innovative technologies to improve global health provides an interesting angle. Much of the agency remains to be filled in, but at least five program managers have been named with experiences ranging from the NCI to the US Navy.
“Transitioning” - a key difference between ARPA-H and NIH projects
Unlike the NIH, the end goal of an ARPA-H project isn’t another grant, rather the establishment of a commercially viable and operating implementation. This will require close coordination with FDA, CMS, VA, CDC, and many other alphabet soup agencies that regulate medical innovation. Perhaps, ARPA-H might be able to shelter its innovations from the ‘valley of death’ that bogs down successful projects in regulatory and commercial market challenges preventing their ultimate productization and adoption.
Would you like to know more?
For a complete summary, read this Congressional Research Summary report, upon which this post is largely based. And of course, the website is found at https://www.arpa-h.gov
🪦 Best of Twitter
Any suggestions for what you’d like to see here instead?
Feeling inspired? Drop us a line and let us know what you liked.
Like all surgeons, we are always looking to get better. Send us your M&M style roastings or favorable Press-Gainey ratings by email at ctrl.alt.operate@gmail.com