tl;dr: first of 7 daily posts from AI Security Bootcamp with 15 other cybersecurity professionals; today set the stage with a survey of AI safety topics and challenges at today’s frontier. Takeaways: securing AI systems is one part of making them safe; cybersecurity mindset applies across safety challenges
As I posted on LinkedIn, this week I’m in Singapore for AI Security Bootcamp 2026. I’ll do informal daily posts – like this one – and follow up with a more formal writeup and takeaways after the week is over. If you want a true tl;dr, feel free to wait for that post to drop.
On the zeroth and first days, I met with friends old and new to settle in and set the stage for the rest of the week: surveying the problem space and where we are now (including a tour of the Claude Mythos Preview System Card), and reinforcing understanding of the AI safety landscape.
View from my bus across town. There is certainly British colonial legacy, but Singapore has built a long way on top. It's also far greener than I expected.
Day 0
Core takeaways: Singapore doesn’t feel as hot as I expected, and feels very much 3-dimensional to navigate; there’s a lot of talent in this room.
AI Security Bootcamp (AISB) 2026 Singapore collected 16 cybersecurity professionals from around the world, plus 3 expert instructors in AI safety and security.
I arrived in Singapore on Sunday and, after meeting with a couple of old friends who now live here (thank you for showing me around!), came straight in on an informal dinner with the instructors and participants, meeting one another and exchanging backgrounds. I’d done prior reconnaissance, but I came away even more impressed with people’s knowledge and achievements.
Day 1: Introductions
Core takeaways:
- AI security - in the sense of securing AI itself - can be seen as a subproblem of AI safety (as good (both moral-good aligned, and useful-good capable) models deployed insecurely, in a broad whole-system sense, can still pose risk)
- The cybersecurity mindset applies to AI safety more generally: while ML people can tend more towards “this will work out, somehow”, cybersecurity practitioners have an eye for “where might this system go wrong”, which transfers even when challenges are not necessarily technical.
- This marries with my experience at Netcraft in particular, where cybercrime can straddle a similar sociotechnical boundary.
The morning was lecture-based, with the outline roughly following:
- more formal introductions than day 0
- an overview of AI safety and security and how they differ
- AI security = securing AI itself, and using AI to enhance security; the former is also part of AI safety, the latter is part of AI capabilities.
- This week focusing on AI security, under the AI safety umbrella
- the tension between capabilities and safety
- safety challenges
- robustness (capability failure)
- misuse (misaligned humans) in particular CBRN and cyber, and loss of control (misaligned AI)
- reward hacking – I liked the visualisation of the reward function and underlying goal as projections onto a 1D space where the reward function has a maximum where the goal had a a minimum
- outer alignment/reward misspecification versus inner alignment/reward misgeneralisation
- instrumental convergence: subgoals that are generally useful (e.g. acquiring money, compute, influence, self-preservation (?)) regardless of what goal is actually being pursued
- capability evaluation; dual-use capabilities
- how do we solve capability uplift for misuse: mass surveillance? strong legislation? narrow-use models “tool intelligence”? what other alternatives?
- This reminded me of a position Bruce Schneier put forward at a talk I attended a few weeks ago (best summarised by reading AI and Trust) with a particular point of difference on public models and empowering individuals – but the focus on legislation very much rhymes.
- existential risk and the black marble thought experiment
- recursive self-improvement
- as an open question: how well do cyber controls apply to AI systems? how well do those cyber controls scale as the AI systems become more capable?
- AI disruption to the economy – labour, capital and technology, where AI might make labour irrelevant – and disruption to power structures balanced on the economy
Discussion was vigorous: we went on numerous tangents providing our own experience, critique, and wider reading across these topics.
In the afternoon, we got our dev environments for the rest of the week set up, and did a tour through the whole Mythos system card, going in depth on select sections. This went a lot further than my reflections-until-3AM on the day the system card came out: I really appreciated the additional perspectives of what else we might want to see in a system card (with the reflection that Anthropic’s model cards are already among the most detailed).
After that, we read through and discussed the training pipeline for modern LLMs: another attendee Shiau Huei did her own great writeup with more details. We focused on OLMo 2 as a fully open source model, and midtraining as an impactful and somtimes overlooked step in particular, using this as a basis to discuss how this data is sourced and affects model outcomes (both pretraining data and how user data is later used), and more generally the shape of training and inference infrastructure (in terms of how compute is shared between training/inference, and the supply chain for the components required).
We finished off with some lightning talks. They were significantly less technical than I expected: I hope we are inspired by the determination to follow through on who killed Captain Alex, and can stay alive for a long time to solve secure AI.
Many of the introductory topics were familiar from the BlueDot Impact Technical AI safety course which I recently completed, but the discussion and shared cyber context added a new layer of depth to conversations. In my immediate BlueDot cohort there were two of us with cyber background – trading depth for breadth – whereas here we all already have this common understanding, but differing perspectives within the field.
It’s exhilarating being far from the smartest one in the room, and I’m really looking forward to learning more from and alongside the cohort for the rest of the week.
If you happen to be reading this before 27th April 2026, please let me know at webmaster@wjbarn.es with any of your thoughts and I’ll be happy to bring them into the rest of the bootcamp with me! If you happen to be in Singapore, I’d be very happy to meet, with the AI Safety x Cybersecurity mixer on Thursday being a particularly good opportunity.