Latest Awards
Issue #37: The Hack That Nearly Hammered Amateur Radio
This month's column deals with a tricky, more technical subject matter involving malicious coding, the Internet, Ham Radio software and more. It may be difficult reading initially, but if you can get through it you'll learn how close we came to disaster -- not only as hams, but as Internet users (And phone users! Both the 'net and our telecom runs on this software.)
Do you know how close we ham radio operators came to potentially being hit with a major hack? For that matter, anyone who uses the Internet could have been stunned by this malicious software trap, one that was only days away from being released to millions of PCs around the world. It was an ingenious act that likely took years to plant when it was accidentally discovered in March 2024 by Andreas Freund, a Microsoft programmer.
This story was related to me by a friend who I interview in this month's column. Michelle Thompson (W5NYV) is co-founder and CEO of Open Research Institute (ORI) in San Diego, a non-profit research institute that does open source digital radio R&D for hams and beyond. Michelle is an Electrical Engineer and her passion for electronics and Amateur Radio lie behind her motivation for the volunteer work she's become known for. She's been an active participant in forums run by the ARRL, IEEE, DEFCON, and many other organizations, as well as QSO Today which is where we became acquainted.
Because the Institute follows the protocols and guidelines of open source development, Michelle is very aware of the intricacies of how that process works and where the potential weaknesses lie. In this case, the "xz vulnerability" came about due to what might be called a "long con" run by an entity, still undiscovered, who called himself/herself "Jia Tan." I will not make an early (and some say obvious) conclusion here about the "State" behind this attack, as anyone can choose any name. Make sure you read the article that Michelle references at the end of the interview. This hack was a rare "10 out of 10" on the scale that security firms use to judge malicious attacks. Suffice it to say that this code, if it had reached the millions of computers it was targeting, would have allowed remote control of affected systems, the input of additional malicious programming, and files downloaded without the users knowledge. In other words, total havoc!
In the interview that follows, Michelle addresses the importance of this find as well as how we might begin to change the open source vulnerabilities that exist today and which allow bad actors to enter the process.
[Dave] - Michelle, what makes this hack different from others, and why should hams be concerned?
[Michelle] - "We should be concerned because 'open source development' powers virtually all of the Internet and a large swath of critical communications infrastructure -- and as Linux is absolutely essential to modern amateur radio, as an operator you've got to care about what happened here. This unquestioned importance of open source work in our life in general is one of the reasons why this incident is so important . . . nearly 100% of Internet infrastructure runs on that software. It's clear that this hack took years and a lot of effort to bring to this point via bullying, power mongering, and targeted harassment. Abuse has become baked-into the open source process. Even ORI projects have been affected by these same issues."
[Dave] - It's particularly concerning that this hack happened in open source, a process that runs on volunteerism in order to benefit us all. To see an attack set up in a system set up for mutually positive collaboration is really discouraging. What are the issues we're talking about here, and will we see this again?
[Michelle] - "I agree. People, out of altruism, motivation and agency, donate their time and talent to make all sorts of open source things. The hack was a costly squandering of goodwill and good effort. It almost worked because individual people with small amounts of power actively exploited the goodwill of many volunteers in order to promote a harmful purpose. It was only two weeks from being integrated into major releases that would have gone to millions of computers all over the world. Most of the big providers of Linux packages were set to include the utility with this hack in their upcoming releases.
Technical work like this is often social before it is technical. If the social framework for the technical work is broken, and I am here to assert that it definitely is, then technical work will be stunted and weakened as we saw in this hack. If you only care about how you're doing and your personal project, then that bad culture might not affect you. But something that is broken and manipulated in your favor will leave others out, which means that your peers may not have an easy time of it; people will drop out and the audience for projects and products will grow weaker and shrink."
[Dave] - In this case, the pressure was apparently put on the fellow who led the project [DGJ note: these leaders are called "Maintainers") through various agents of the entity desiring to manipulate the work being done on this utility software. I've read that the fellow was bombarded with complaints from other users -- who have all now disappeared -- about his tardiness in moving the project along. The pressure increased until he gave in, allowing another leader to emerge. It was this Jia Tan as Co-Maintainer who later brought the "backdoor" hack to the xz project.
[Michelle] - "That's right, the xz Maintainer was targeted and manipulated in a way that's become totally acceptable in open source work. The terrible way that this volunteer and project leader was treated has been normalized. At ORI we have spoken up against this sort of manipulation in the past and will continue to speak up against it as long as it is a problem.
Unfortunately, in some cases the mentality that led to the xz hack is rewarded and reinforced. Sadly, we see this occurring in amateur radio circles as well. Attacking anyone that might be a "threat" and isolating/targeting the people who want to be collaborative and productive . . . these are actions taken out of jealousy and spite. When bad behavior is found in our hobby it harms amateur radio. These views are unfortunately very real and are widely held. It can hurt us because people take actions based on their unconfronted views and prejudices. The way that the xz hack has hurt the Linux community resonates for me because this bad culture harms individuals in amateur radio in much the same way."
[Dave] - Michelle, you made a transition here from the xz vulnerability hack to amateur radio, by drawing a comparison to the way that open source development proceeds. Can you give me an example of what you might be talking about with ham radio specifically?
[Michelle] - "One example I can give would be the negative effects on amateur radio that stem from Amateur Radio Digital Communications (ARDC). ARDC converted a community asset set aside for amateur radio -- Internet IP addresses -- into a very large, privately controlled fund. The conversion of those amateur radio IP addresses into cash was very much like a hack in and of itself, as the sale was OK'd only by a tiny number of ARDC directors representing the hobby. It was done without any public notice or discussion. Instead of holding those addresses in trust for the amateur radio community, the ARDC was suddenly armed with a boatload of cash to 'fund worthy projects.'
Instead, they bullied, interfered with, and harmed those projects. Multiple members of the ARDC board have threatened individuals with being 'cut off' from any future funding for perceived slights, insults or defects. Applicants have had their grant requests arbitrarily deleted or significantly modified. In one case, a board member disagreed with a post made on social media and threatened the target of their ire with a permanent ban. In another case, a volunteer commented in a forum post about ARDC making a backroom deal and was then targeted. In general, ARDC money has caused a significant shift in the way that players in amateur radio have behaved towards each other. Without a serious course correction, the damage will continue. We don't need a group of 'venture capitalists' playing politics with our radio projects . . . the amateur community desperately needs safe and fair engagement. Dave, the bad culture you see in the xz hack we've been discussing is being repeated in our hobby."
[Dave] - After hearing your views on this, Michelle, I have the feeling that the culture of abuse in a system that uses volunteers who "care" is going to be very hard to change. I've never been involved in open source development, but I know how many interesting amateur radio projects have come up this way. Please share more about the bad culture.
[Michelle] - "Broadly, it's an ethics problem. It's deeply unethical to simultaneously define 'critical internet infrastructure' as also 'just a hobby done in your free time.' But we do this. People do amazing things under bad circumstances when they clearly see an unmet need. In this way, society gets something for nothing by burning people out. We let volunteers stick their necks out, work hard, and publish extremely useful results of all sorts and then we fail to back them up. In the case of open source maintainers as in the story behind the xz vulnerability, the people most often taken advantage of are those that act supportive and nurturing. Blowhard bullies get a pass, no matter how absent or infrequent their work, but people who altruistically care are a relatively easy target. They may be ghosted . . . made to feel inferior in whatever way is convenient to keep them silently productive.
[Dave] - It's amazing that it works as well as it does. The fact that open source work creates anything of value with such inherent issues is just amazing.
[Michelle] - "While that's true, and we have a broad wealth of open source work, eventually corrupt code will be published in the supply chain as a result of this cultural dichotomy of value. The missing ingredient? Donated open source work needs to be properly valued or it will be socially exploited, just as it was in this particular case. Money can't save the day, even though funding does help -- but it helps only if it is properly administered. As in my radio example, we've seen that money can ruin things when it is not. Financial support doesn't help if the social bits are still broken.
Beneath the surface, the issue is a repeated insistence by larger society to relegate open source work as unpaid labor that is "owed" because "that's your job, so shut up and do it really well." Treating people badly is a security risk, and failing to value meaningful work gifted to the general public is equivalent to treating people badly. This isn't hard to figure out. But it has been incredibly hard to fix, even when the costs are painfully clear.
The bad actors in the case of the xz hack (and you're right, this was a long con hack) were able to do this with impunity because they used the same nasty tactics that *work* in many open source projects. Shame and doubt and insinuations that the maintainer is alone and a failure . . . sadly, this is common."
[Dave] - Had this hack worked, amateur radio operators would have been severely affected, but we would have been only one tiny downstream element of the damage done. The Internet, though, is such a big target. No wonder this much time was invested on the part of the entity that attempted to pull it off. The confusion and damage, financial and other, would have been huge. Do you see any coming changes that might mean we can catch these in the future? And for this audience, any changes in the way that amateur radio projects work in the world of open source?
[Michelle} - "Protecting the Internet is super important. The Internet should "just keep working." The overwhelming majority of the Internet runs on open source software. The long-term effects on society of devaluing the workers, while expecting the work to be done for free, have been studied and are quite negative. You can literally see the contempt for open source maintainers in thousands of github comments. This ugly fraction of open source participants echoes strongly in amateur radio, a culture that is extremely homogenous and very resistant to change. Our modest efforts at ORI, our attempts to be competent, selfless, collaborative, and supportive, have been run roughshod at times. I can assure you that working hard and watching incredibly talented technical volunteers getting kicked in the teeth as a "reward" is not fun at all.
We'd have access to more spectrum and be much more deeply involved in the regulatory process if we were socially healthy and inclusive. But becoming socially healthy and inclusive requires us to enforce repercussions for bad behavior. Without that key ingredient, no amount of role models, representation, publicity, or money will help."
[Dave] - Thanks for an interesting conversation, Michelle. You've certainly given us some things to think about. I hope the forum attached to your commentary here will bring us more opportunities to debug these issues as they relate to the amateur radio services.
[Michelle] - "Being honest about addressing the social problems of ham radio is the key to solving nearly every other problem in the radio service. We can continue to deny this fact and barely survive with flat or negative growth in the US while watching other countries experience steep licensee declines, or we can change course and take full advantage of the best it's ever been for the radio arts. What are we waiting for?"
[DGJ] Michelle suggests the article at this link for a well-rounded explanation of how the xz hack came about. I hope to see you all on the attached forum discussion!
73 for now,
Dave Jensen, W7DGJ
Michelle Thompson, W5NYV
Have a comment? See what others are saying now in our Forum discussion! CLICK HERE and JUMP INTO THE CONVERSATION
There's plenty to talk about on the Forum discussion. See this link (bottom post on that page) for one ARDC director's views about the article and commentary from MIchelle.
Dave Jensen, W7DGJDave Jensen, W7DGJ, was first licensed in 1966. Originally WN7VDY (and later WA7VDY), Dave operated on 40 and 80 meter CW with a shack that consisted primarily of Heathkit equipment. Dave loved radio so much he went off to college to study broadcasting and came out with a BS in Communications from Ohio University (Athens, OH). He worked his way through a number of audio electronics companies after graduation, including the professional microphone business for Audio-Technica. He was later licensed as W7DGJ out of Scottsdale, Arizona, where he ran an executive recruitment practice (CareerTrax Inc.) for several decades. Jensen has published articles in magazines dealing with science and engineering. His column “Tooling Up” ran for 20 years in the website of the leading science journal, SCIENCE, and his column called “Managing Your Career” continues to be a popular read each month for the Pharmaceutical and Household Products industries in two journals published by Rodman Publishing. |
Articles Written by Dave Jensen, W7DGJ
- Trials and Errors #46: One Remarkable XYL - November 18, 2024
- Short Takes #29: Sweeteners and Buzz-Killers - November 3, 2024
- Trials and Errors Issue #45: General Electric and the Wild West of Early Radio - October 8, 2024
- Trials and Errors #44: Copycats Stomp on Ham Radio Innovation - September 15, 2024
- Trials and Errors Issue #43: Building Out the Antenna Farm - August 29, 2024
- Short Takes #28: Mastering the Waves -- A Primer on Magnetic Loop Antennas - August 5, 2024
- Issue #42: Traits of Our Radio Innovators - August 4, 2024
- Short Takes #27: Costco Ham Shack Item, Remote Testing, and a Hero Code Warrior - July 9, 2024
- Issue #41: How Do New Operators View Amateur Radio? - July 3, 2024
- Issue #40: When is a Ham Radio NOT a Ham Radio? - June 15, 2024
- Short Takes #26: Cables, Coax, Connectors and Kooks - May 14, 2024
- Issue #39: Forecasting the Future of Ham Radio - May 13, 2024
- Issue #38: The Boys of Radio - May 2, 2024
- Issue #37: The Hack That Nearly Hammered Amateur Radio - April 10, 2024
- Issue #36: Ham Detective Solves 150 WPM CW Mystery! - March 8, 2024
- More articles by Dave Jensen, W7DGJ...