As someone who has built a career on Kubernetes, I'm always thinking about what "the next platform" is (for job security purposes). By "platform", I mean the kind that platform engineers like myself build for internal dev teams on which to run their applications. Kubernetes has been the star platform for quite a few years now (or maybe that's...
This post will make more sense if you first read Dear Sir, You Have Built a Compiler. Dear friend, I am afraid to inform you that you have built a Kubernetes. I know you wanted to "choose boring tech" to just run some containers. You said that "Kubernetes is overkill" and "it's just way too complex for a simple task" and yet, six months later,...
Recently at work I've learned about Cloudflare Tunnels, which has increased my interest in tunneling technologies in general. Tunneling is a generic networking term, but in the web-dominated world it usually refers to software that you can run on any computer with outbound network access to serve a website. The tunneling software has a another...
If you're reading this blog, there's a high chance that you are one of the 1-in-5 workers who has a non-compete clause hiding in your employment contract. These are clauses that prevent workers from taking other jobs or starting other businesses that "compete" with the employer after leaving the job. I originally thought they were unique to...
Lately I've been obsessing over this ProPublica article which talks about Microsoft's involvement in the SolarWinds hacks, specifically due to a design weakness in Microsoft's Active Directory Federation Services (ADFS). It's a great article, highly recommended. Anyway, the wiki page for ADFS says it first appeared around 2003, which made me...
Back in January SourceHut and Codeberg both experienced a distributed denial of service (DDoS) attack. That attack made me worried about the future of the small, independent Internet: services like SourceHut and Codeberg which do not use the oligopoly of large hosting providers. DDoS attacks are where an attacker controls a bunch of computers...
Last year I left a comment on HN about Docker Compose that seemed to get some traction. I was complaining about how both Docker Compose and Kubernetes are leaky abstractions over infrastructure. My general stance is that the Kubernetes API, despite its vast complexity, allows you to describe like 99.9% of modern applications. All attempts to...
I wanted to write this because I don't hear enough real people discouraging the use of Web Application Firewalls (WAFs). Probably because the search results for "Web Application Firewall" are all written by WAF vendors. Anyone reading just that could conclude that WAFs are a good idea. I'm here to offer another perspective, after having suffered...
I recently picked up a few books to read, and Security Chaos Engineering engrossed me so significantly that it 1) made me read the whole thing and 2) made me want to write about it too! If you just want a taste of the book, check out the primary author Kelly Shortridge's blog. Her unique writing style is quite fun to read. My interpretation of...
This is a story about how in 2021, I discovered a vulnerability affecting an unnamed billion-dollar company and disclosed it to them, earning my largest bug bounty ever! Accidental Discovery At first, I was just doing some work on kubecost, a Kubernetes tool which estimates costs for running pods. The first thing that caught my security eye was...
Back in 2020, A Russian state-sponsored group got into SolarWinds' build system and inserted command and control (c2) code into a routine software update for a network monitoring tool called Orion (wiki link). It was all over the news, and for good reason given the extent of the breach (into particularly sensitive parts of the US government) and...
Unlike many other industries like doctors, real estate agents, lawyers, social workers, etc. the tech world doesn't have to abide by any particular code of ethics. We can argue all we want about if such a thing should be required or what should be in it, but I think clearly something is better than the nothing we have today. In most of the US at...
If you didn't hear, CircleCI recently released their report on a December 2022 security incident. What stood out to me is that CircleCI seems to be doing all the "normal" security things: SSO with 2FA, endpoint protection, auditing/logging, encryption at rest, etc. But they still got breached. I think this speaks to the unfortunate state of the...
Take a second to think through all the "stuff" you have to personally maintain at your job. Here are a few ideas: Physical servers or virtual machines Internal libraries A handful of microservices Some test cases you wrote CI/CD stuff Helper scripts Open-source repos Security/legal compliance Software licenses All of the tools on your computer...
Securing Kubernetes is complex, so there are quite a few guides out there: CIS Benchmarks for Kubernetes (free account login required) NSA/CISA Kubernetes Hardening Guidance Kubernetes Pod Security Standards Various, other, sources Unfortunately, following all those guides and patching every CVE still might not be enough. There are some security...
In 2016, Kubernetes v1.3 was released which included a new API type: PodSecurityPolicies (PSPs). The original design proposal had the lofty goal of allowing cluster admins to restrict various Linux privileges to some Pods while still allowing other Pods to use them. PSPs filled an important security hole where having the ability to run pods could...
It's no secret that Kubernetes Secrets are just base64-encoded strings stored in etcd alongside the rest of the cluster's state. Ever since the introduction of Secrets in 2015, armchair security experts have been scoffing at this decision and seeking alternatives. I think those people are missing the point. The design of the Secrets API dates...
Since cyber attacks, a building collapse, and an infrastructure bill have been in the headlines recently, I think the time has come for us to re-assess our relationship with maintenance. While I work in the "tech" field, I don't align myself with inventors, entrepreneurs, or green-field software engineers. I align myself with the whole spectrum...
To those of you who give advice on which languages, methodologies, architectures, or whatever else to use, I want to start by saying thank you. Your advice is vital to the growth of millions of people like me who are early in their careers. But I've noticed a pattern with software engineering advice that leads to over-engineering and unnecessary...