One of my projects that runs 24x7x365 is a web scraper. I’ve iterated on the approach several times, and what is currently working is to do the fetch and then save the entire HTTP response in a Postgres database. From various other projects, I can then query one central database and extract out data that … Read more
Yesterday I got hit with the double whammy. I tried to upgrade GitLab to v16.3.4 as part of their recent critical security release when I realised that the GitLab server was running version 15.x (I don’t remember which minor release), so first I would need to install 16.0.0, then get up to 16.3.4. This, it … Read more
This one cost me a whole bunch of time. Annoyingly I got a Docker version of Postgres 16 working on one machine, but a couple of days later I needed to update another and I hit this error: Somewhat fortunately for me, this time around, was that this was a test setup I have used … Read more
Recently I have been doing some web scraping in Kotlin, and have now got a regularly updated set of records appearing in my database. An interesting challenge arose when the scraped data was not quite as nice to work with as the web view I was scraping from. With a little front end whizz-bangery, the … Read more
Here’s the problem we are addressing today: you have a set of numerical data returned by your Postgres query. Your numbers look alright, but some very high (or very low) values seem to be throwing things out of whack. What you’d like to do is eliminate the outlying values from your result set, and keep … Read more
Bit of an odd one today. I was browsing around a production system (my own, thank you), and tracing my way through some Postgres routines trying to track down why a query seemed to be returning an unexpected / outdated result. When I opened one of the routines I was met with this: The real … Read more
Today I needed a query that could take some nested JSON data, a column ID, and then create a single row for each of these distinct column ID’s to show some summary numbers of different nested JSON values. I’ve horribly explained that, and so you can imagine just how much fun I had writing the … Read more
I’ve come across the select case syntax in Postgres before, but never had to figure out what it does. However on my current project I was faced with a query where I had to understand it, so I went about setting up a little local demo file to figure out how to use case in … Read more