Jason Bamford's blog

One Hour PNG Development - Image Encoding in Python

In this post, we look at rapidly prototyping a PNG image encoder in Python.

The aim is to create a PNG encoder that does not rely on any code not available within the Python standard library, and in doing so learn a bit about the internal structure of the PNG format.

You should already be familiar with what a PNG file is and with basic image transmission concepts.

All of the code required is given in this post, however a repository of the complete program can be found on GitHub.

Format Summary

First let's summarise some information on the PNG format specification, mostly found by reading the standard at www.w3.org/TR/PNG/

The minimum mandatory chunks are a header (chunk type "IHDR"), image data ("IDAT"), and an image trailer ("IEND"), appearing in that order.

Chunk and Data Formats

Table 1. Chunk Format
04Length - number of bytes in data field
44Chunk Type - four ASCII alphabetic characters
8nData Field
n+84CRC check value

IHDR Header Chunk

Table 2. Data field contents for IHDR chunk
04Width - image width in pixels
44Height - image height in pixels
81Bit Depth - bits per colour plane per pixel - 1, 2, 4, 8 or 16
91Colour Type - 2 = Truecolor RGB with 8 or 16 bits per colour per pixel
101Compression Method - always 0 [3]
111Filter Method - always 0 [3]
121Interlace Method - 0 = none, 1 = Adam7 interlace

IDAT Image Data Chunk

The data field contains the compressed image data as output by the compression algorithm.

IEND Image Trailer Chunk

This marks the end of the file and has an empty (zero length) data field.


PNG uses an LZ77 based compression algorithm called Deflate.

Working in Python 3 allows us to hand-wave over the details of the compression algorithm [4], as Python provides an implementation called zlib within it's standard library. [5] Conveniently, the zlib library is even able to calculate CRC values for all of our chunks for us.

Writing an Encoder

We will be following the common code pattern defined by use of a main() function. Since we are using the zlib library we are sure to import it at the top of our file. Our first piece of code creates a bytes object holding the signature values defined in the standard.

import zlib

def main():
    signature = bytes( (137, 80, 78, 71, 13, 10, 26, 10) )


if __name__ == "__main__":

Following tables 1 and 2 above, we create a function to take any data and build it into a chunk, and another to generate the data for an IHDR header chunk.

def make_chunk(type, data=b""):
    b = len(data).to_bytes(4, byteorder="big")  # Length of Data
    b += bytes(type, "ascii")                   # Chunk Type
    b += data                                   # Data Field
    crc = zlib.crc32(bytes(type, "ascii"))
    crc = zlib.crc32(data, crc)
    b += crc.to_bytes(4, byteorder="big")       # CRC
    return b

def make_header(width, height):
    b = width.to_bytes(4, byteorder="big")    # Width
    b += height.to_bytes(4, byteorder="big")  # Height
    b += (8).to_bytes(1, byteorder="big")     # Bit Depth
    b += (2).to_bytes(1, byteorder="big")     # Color Type = Truecolor
    b += (0).to_bytes(1, byteorder="big")     # Compression Method
    b += (0).to_bytes(1, byteorder="big")     # Filter Method
    b += (0).to_bytes(1, byteorder="big")     # Interlace Method = None
    return b

We can now add a call to these functions into main() to show them working.

def main():

    chunk_ihdr = make_chunk("IHDR", make_header(width=4, height=4))


Running this will output a string of gobbledygook, but if you examine it closely you will find it corresponds to the data structures defined above - note the two instances of "\x04" denoting the width and height, and "\x08\x02" for the bit depth and colour format bytes.

Read full article...

Django Minimalist Custom User Model

In this article I will look at creating a minimal custom user model in Django.

The aim is to create the simplest possible user model, providing only a username field and a password field, that will allow logging in, logging out and testing authorisation with the standard Django auth interface.

Being as small as possible is the goal at all costs. The user model we create here will not be suitable for accessing the Django admin site, as the admin requires fields that we will be leaving out.

This user model is unlikely to be usable as it stands, but provides a clean slate on which a customised user model could be built, without clutter in the background.

Research - AbstractBaseUser

Online sources including the official Django documentation suggest we may want to build our custom user model on top of the AbstractBaseUser class. We will begin there, and it is instructive to first examine the code in the libary for AbstractBaseUser.



class AbstractBaseUser(models.Model):
    password = models.CharField(_('password'), max_length=128)
    last_login = models.DateTimeField(_('last login'), blank=True, null=True)

    is_active = True



Examining AbstractBaseUser in the django library, a password field is already provided for us, along with a last login field, a fixed is_active property, and the REQUIRED_FIELDS empty list)

Coding Our Custom User Model

In this example, we have named our custom user app 'customusers'.

We will be writing our classes in 'customusers/models.py', building on BaseUserManager and AbstractBaseUser, so we include these at the top of the file.


from django.db import models
from django.contrib.auth.models import BaseUserManager, AbstractBaseUser

The User Manager

Our custom user model will require a custom user manager, whose only required purpose is to handle creation of users. The user manager is set as the 'objects' property on the user model, and inherits from BaseUserManager, which in turn inherits from models.Manager, therefore inheriting the usual database management methods.

Django passwords are stored as a hash and therefore need to be set using the set_password function (provided by the AbstractBaseUser class).

The create_superuser method is required so `manage.py createsuperuser` will work.


class CustomUserManager(BaseUserManager):
    def create_user(self, username, password):
        user = CustomUser.objects.create(username=username)
        return user

    def create_superuser(self, username, password):
        user = self.create_user(username, password)
        return user

Read full article...

Python 3 Venv - Local and Central Virtual Environments

Since version 3.3, Python includes the module venv which allows you to create isloated environments, where you can install any packages, libraries etc. that your project may need, without cluttering or affecting other projects on your system - you can have as many separate environments as you like.

Since venv comes as part of Python it is available to use at all times without requiring installation.

Using Venv

First I will show you how to create a virtual environment in your project folder, as most tutorials do. But please then read on as other possibilities abound!

The following assumes you are working on MacOS or Linux using the bash shell.

Starting in the folder for your python project:

If you usually have to call Python with the command `python3`, while the virtual environment is active you are able to use just `python` instead.

You can also run your virtual version of Python without activating the virtual environment, by referencing the binary in the .venv directory directly, eg.

$ .venv/bin/python

Centralised Virtual Environments with Venv

Tutorials I have seen on venv only show you how to create a virtual environment in the project folder you are working on, but this does not need to be the case at all.

You can just as easily create a virtual environment in a centralised location and use it while working on multiple projects. For example, if you have several Django projects you're working on, you don't need to have a virtual environment in each separate project, all with the same version of Django installed; just create one environment in a convenient location.

You might for example choose to keep central virtual environments in a subfolder of your home directory such as ~/.venv/

Following the same instructions as above (and naming our environment django as an example):

Read full article...

Pure CSS Collapsibles and Accordions

CSS Collapsibles Can Be Easy and Even Classless

This is how you do a pure CSS collapsible, using the usual checkbox trick, but without all the extra nested <div> containers and one million classes.

See towards the bottom of the page for how to expand this into an accordion.

Overview And Full Code

Three html elements are needed for a collapsible. A checkbox <input>, a <label> and a <div> [note 1]

An overall outer <div> is needed when the collapsible is to follow a non-explicitly closed <p> [note 2]

The html for all our collapsibles is:

<div>  <!-- outer div not always necessary -->

  <input id=collapsible-id class=collapsible type=checkbox checked>
  <label for=collapsible-id>Collapsible Heading</label>
    Collapsible content.


And the full CSS:

input.collapsible { display: none; }

input.collapsible + label {
    display: block;
    width: 60%;
    background: #0070ff;
    color: White;
    text-align: center;
    padding: 1em;
    cursor: pointer;
    transition: all 0.5s ease-out;
    /* dynamic */
    border-radius: 4px;

input.collapsible:checked + label {
    /* dynamic */
    border-bottom-left-radius: 0;
    border-bottom-right-radius: 0;

input.collapsible-demo-2 + label::before {
    font: bold 1em sans-serif;
    margin-right: 0.5em;
    /* dynamic */
    content: '+';

input.collapsible-demo-2:checked + label::before {
    /* dynamic */
    content: '\2212'; /* full width minus */

input.collapsible + label + div {
    overflow: scroll;
    width: 60%;
    background: #0070ff44;
    border: none;
    transition: all 0.5s ease-in-out;
    /* dynamic */
    max-height: 0px;
    padding: 0 1em;
    border-radius: 0;

input.collapsible:checked + label + div {
    /* dynamic */
    max-height: 100vh;
    padding-top: 0.5em;
    padding-bottom: 0.5em;
    border-bottom-left-radius: 4px;
    border-bottom-right-radius: 4px;

Read full article...

Acorn BBC Model B Faulty RAM Repair

About This Machine

The BBC Micro was a Microcomputer made by Acorn Computers; introduced at the end of 1981 as part of the BBC Computer Literacy Project, it was succesful both as home computer and in the educational market, with most schools in the UK using the machine for computing education. During the 1980s and 1990s, using a BBC Micro in primary school was the first experience of a computer for many people - including the author.

The Fault, Testing and Research

This example, despite being in immaculate physical condition, really wasn't happy when turned on; it seemed to put out a very bad video signal. Sometimes it appeared it was trying to work, but the picture would judder wildly.

Testing with an oscilloscope revealed the video signal looked ok, but on closer examination the vertical sync rate was 80Hz, obviously not to the liking of the 50Hz monitor!

A little research and head scratching later I figured out this is symptomatic of a RAM fault, causing the CPU to read garbage values that then get written into the 6845 video controller during boot, resulting in the weird output frequency. Further I came across a trick of moving a jumper on the motherboard (S25) to disable half the RAM, and this indeed worked, the machine operating happily and reporting itself with 16k of memory. [1]

Read full article...

Arduino - Getting Off The Dev Board

The Arduino Uno is a readily available development board for the Amtel ATmega328 microcontroller, allowing the user to easily experiment and iterate with their design. For use in an actual product, or to make a one off gadget even smaller, it is advantageous to use the microcontroller chip independent of the Arduino board. These notes look at operating the ATmega328 by itself on a breadboard, and how to program the chip in situ using an Arduino.

Advantages of Bare Microcontroller


Simplified ATmega328 pinout showing Arduino pin names

Minimal ATmega328 Setup

Minimum setup of ATmega328 with no crystal and single LED for output.

Read full article...

Alert Dialogs in Android

In this post I will look at the basics of making alert dialogs appear on Android.

An example Android Studio project demonstrating the code shown here is available on GitHub.


Android provides the AlertDialog class for creating dialog boxes with a simple structure of a title, content (typically a text-based message), and up to three action buttons.

An instance of AlertDialog is created (using the provided Builder class) and various setter methods are used to create our design, which can then be shown with the show() method.

Methods of AlertDialog

Builder(Context)Nested class, who's create() method returns a new AlertDialog object. Context parameter should be the current activity object, eg. MainActivity.this
setTitle(CharSequence)Sets the title text for the alert dialog
setMessage(CharSequence)Sets the message text
setButton(int whichButton, CharSequence text, DialogInterface.OnClickListener listener) Sets the button caption and listener to be called when button is pressed
setCanceledOnTouchOutside(boolean)When set to true user can dismiss dialog by touching elsewhere on the display
setOnCancelListener(DialogInterface.OnCancelListener) Set listener to be called if dialog is cancelled by touching outside
show()Causes the alert dialog to appear and take focus
dismiss()Make the dialog go away and return focus to the activity

Read full article...

PHP Built-in Server

PHP has a really useful feature, which is a built in web-server, great for local testing of a php website. It's all installed and available by default on MacOS and some versions of Linux. To use, cd to the html root directory and run:

$php -S localhost:8888 testserver.php

This will run the file testserver.php for each request received from the local machine at port 8888.

Example testserver.php file:


/* remove get parameters */
$file_uri = explode('?',$_SERVER['REQUEST_URI'],2)[0];

/* remove leading / to make relative unix path */
if ($file_uri[0] == "/")  $file_uri = substr($file_uri,1);

/* run controller to handle non-existent uri */
if (! is_file($file_uri)) {
  include 'controller.php';
} else {
  /* all other files served automatically by php server */
  return False;

PHP Manual - Built-in web server

Read full article...

KHP-30 High Voltage Probe

The KHP-30 is a high voltage probe with a built in meter, measuring up to 30 kV DC. It is meant primarily for testing the anode voltage on CRTs.

The box indicates this was made in Japan, and based on some of the materials used in the probe I would say it dates from the 1970s. This example is branded 'Eagle Products', but I have seen otherwise identical looking probes with different brand names online.


Input Impedance   600 Meg
   Current Draw   50 uA at 30 kV


All resistors have 5% tolerance. The adjustable leg with R3 and the preset resistor takes about 10% of the current and can be adjusted to compensate for the tolerances of the resistors and the meter movement.

My meter movement was full scale at 43uA and measured 3,200 ohms. The preset resistor was set for a resistance of about 57k. For details of the "ground ring" see the constructional information and photographs below.

I reckon the probe is built to withstand continous operation, a full scale reading would only dissipate 1.5W, however with my example after 5 minutes or so measuring 22kV the reading slowly rises about 5 percent, as the large resistors do warm up a little. The reading returns to normal after giving it a few minutes to cool.

Read full article...