I think I’ve heard people say data is the new gold at least twenty times. Having worked in data for a while, I’m pretty sure what they mean by that is that extracting and processing is a laborious and often violent process. It’s possible that my take is inspired by working as a data engineer, and therefore being the victim of other people’s data...
I’ve been working hard on a new project called Wimsey lately. I’ll save writing about the project itself for another day, but it got me thinking about github actions (or really any CICD workflow tool). And crucially, how my process for setting up a new CICD flow always winds up being: Write a nice clean yaml config Feel like everything is very...
I don’t keep it much of a secret- I love functional programming. Or maybe I’m just burned form spending hours of my life chasing back inheritance to see where an object variable was defined. Either was, when I saw that hy lang v1.0 was release the other day, I was pretty kean to try it out! One of the downsides of new or more experimental...
Especially in the dark realm of data engineering, there’s a huge range of neat low-code/no-code UI tools. I don’t want to complain about those today, but I do want to talk about why libraries (as opposed to low-code UI) are really awesome. Low code is good code! One thing that I think get’s missed out, is that low-code can still be code. Plotly...
I don’t know where abouts we sit on the wave of yaml-domain-specific-languages. I really hope it’s the peak, and that things will simmer down. I like yaml a lot as a configuration language, but every time I was to work in a domain specific language pretending to be configuration, it sends chills down my spine. A big part of it is that github...
People (sadly) don’t go around asking me my favourite text editor a lot. But if they did, I’d answer in a second, (“Helix!”)[https://helix-editor.com/]. But weirdly, I have worked a bunch in the past with people who are new to coding, and have recomended them editors, I’ve never recommended Helix to someone starting out. That probably isn’t that...
The other day, I listened to the episode of Richard Feldmans’s Softward Unscripted with Greg Wilson. It’s a brilliant episode (in fact, the every episode is brilliant so you should probably check it out instead of reading this) and Greg has a lot of great insights around how we teach and discuss software engineering. One thing that really got me...
I love taking some time out in December, but my code obsessed brain often results in me using it to learn about some new coding concepts. This year, I took to reading though Miran Lipovača’s awesome Learn You a Haskell for Great Good! which had previously sat on my bookshelf unread for a long time. Probably one of the main reasons I didn’t...
Bemoaning the design of pandas isn’t a very hot take - the success of polars which seems to have kinda rocketed out of nowhere, is probably down, not by a small amount, to just how many people are currently using pandas, but want it to be faster (always better right?) and also more elegant. I’m not gonna say to use pandas over polars or...
I should probably admit that there’s a bit of a contradiction between two thoughts that I have: I really love spark I really hate spark Spark is one of the most powerful dataframe libraries on the planet. It can process multiple petabytes of data. But it’s also overkill and unwieldy for most jobs. For smaller datasets, tools like Polars or Duckdb...
I have a pretty unusual and specific guilty pleasure. Well actually I have an entire music taste based on that premise. More specifically though, I have one that relates to programming: clean coding. By that, I don’t mean that all my code is spotless or beautiful or anything, but some part of my brain really enjoys reading about clean code. I...
Good news if you work in software - impress your friends and colleagues with these magnificent sentences! “Have we thought about scalability?” “Let’s stop and think about what problem we’re actually trying to solve” “We don’t need to reinvent the wheel” You can say these at any point, in any meeting, without considering context and people will...