what is vdo?

virtual data optimizer. it adds deduplication support to logical volumes created using lvm. well not vdo itself, but a sort of plugin for lvm that exploits the dm_vdo kernel module

why dedupe?

i want to play around with libvirt and install a bunch of vms but i don’t want to waste storage space

will this work?

honestly, no clue. i don’t think so. i think while lvmvdo is cool, libvirt doesn’t support using predefined lvm logical volumes as libvirt storage volumes. i saw there was a way to define an lvm volume group as a libvirt storage pool. and then libvirt is supposed to create logical volumes from said libvirt pool/lvm volume group as needed

but for all this vdo to work, i need to define the volumes outside of libvirt. can i tell libvirt to use a specifc lvm logical volume? actually yes. i don’t need to define the storage medium as a libvirt volume hmmm. ok but that’s for another day

how does lvmvdo work?

uses a “pool” a vdo pool to store the actual data, and the volume group that’s available as a block device for writing a filesystem onto is a sort of thin volume. so applications see the filesystem on a virtual block device. the virtual block device is writing data to the “pool” and the pool is deduping and also compressing as it receives data

how do i want to use it?

i want a single vdo pool to be shared across multiple thin volumes

why would i want that?

vms would have similar data. duh

how do i do that?

at first i was looking at the docs on lvmvdo(7) but that was confusing. turns out the instructions on lvmthin(1) are simpler

# lvcreate --type thin-pool -n <poolname> -L <size> --pooldatavdo y <vg>

so this automatically create a vdo pool on <vg> a then a thin pool on the vdo pool

another thing

the automatically created vdo pool is called vpool. so that ends up being confusing because now i see vpool0_vpool in the output of lsblk. so just for my convenience

# lvrename <vg>/vpool0 vdo

creating the thin volumes

well almost all everything is done, i still need to create thin volumes that can be used as block devices

# lvcreate --type thin --thin-pool <poolname> [-n <thinvolname>] -V <virtualsize> <vg>

leaving out the -n <thinvolname> gives me a logical volume called lvolN where N is the next number in the sequence of logical volumes. not bad, but not ideal. i should use the -n flag

using thin volume that’s on a thin pool that’s on a vdo pool

create a filesystem

# mkfs /dev/mapper/<vg>-<thinvolname>

and then do whatever. it’s not apparent to me/you nor the applications that there are so many layers of abstraction under this virtual block device called <thinvolname>.

<thinvolname> has its blocks allocated dynamically from a thin pool. and the thin pool in turn is sitting on a vdo pool that’s deduplicating and compressing the blocks written to it

max space savings hopefully. next step is figuring out how to actually use this with libvirt

This one time, a classmate asked me after class “What are you always coding?”

I was simply taking lecture notes in vim.

Okay, most would immediately disqualify my being “plebeian” on account of my using vim… doesn’t help that I was writing in markdown.

But this post is more about the terminal being useful in fields other than programming. In this particular case, a university literature course.

The splitting task

The instructor was going to go over excerpts from course material. Some of the material was really long pdf files. But we only needed a few sections. How do we extract out only the parts we need?

We could pay Adobe to let us extract pages. Or my sneaky trick of opening the pdf file in a browser and printing to file after typing in the range of pages I need. But repeating that 12 times with different page ranges each time is meh.

The CLI can split and unite

On the terminal, I have access to a tool called pdfseparate that extracts pages from a pdf. But each page is saved to its own file. Luckily, there’s also a pdfunite that does I could use to combine these files.

But while that solves the question of tools to use, I do still need to repeatedly specify the page numbers somewhat like

pdfseparate -f 4 -l 5 "filename.pdf" "filename-p%d.pdf"
pdfunite "filename-p4.pdf" "filename-p5.pdf" "filename.excerpt.pdf"

That is two separate commands for just one file and each filename and page count needs to be stated so expressly… how is this more efficient?

It isn’t. But the command line offers so much opportunity for efficiency through other features such as loops.

Loops

I could simply change my command to go over all the pdf files and run the commands for each one. By using a variable ${file}, I can let the terminal (technically shell) change the filename for each command.

for file in *.pdf; do
    pdfseparate -f XYZ -l XYZ "${file}.pdf" "${file}-p%d.page" ;
    pdfunite "${file}-p*.page" "${file}.excerpt.pdf" ;
done

Additionally, I set the individual pages to be saved to .page files instead of .pdf file. This lets me use the ${file}-p*.page part to indicate a pattern of filenames instead of specifying each file that needs to be combined by pdfunite; and also won’t interfere with my loop that is looking for the pattern *.pdf.

Wildcards / Globs

The * is called a glob character (or wildcard) to mean that it can match anything. In this particular case, it is being used to select all files whose names match a particular pattern.

Varying page numbers

Yet persists the problem of the required page number ranges being different for each pdf file. I could use variables again. But for this one, a little bit of manual typing is needed. So I created a spreadsheet and pushed off delegated to a classmate, the work of keying in the first and last page number for each filename.

Spreadsheets can be easily exported to CSV (comma-separated value) files. Then, I used a different loop on the CSV file’s rows instead of working on *.pdf as I did before.

filename-093.pdf,5,45
fileplace-02.pdf,3,12
fileanima-98.pdf,9,11
filethgns-09.pdf,1,9

The above is a sample CSV file. Now I could get values for three variables from each row of the file, the filename, first page and last page.

while IFS=, read file firstp lastp; do
    pdfseparate -f ${firstp} -l ${lastp} "${file}.pdf" "${file}-p%d.page" ;
    pdfunite "${file}-p*.page" "${file}.excerpt.pdf" ;
done

You could also go further and combine all the excerpts into a single big file using the handy *.excerpt.pdf pattern.

pdfunite "*.exercpt.pdf" "allmaterial.pdf"

CLI for mooore than just code

And with that I conclude this badly written blog post. The CLI is useful in situations other than coding—such as a university literature course.

Or to simple take notes in class if you use vim 😎

for name in *.txt; do
    mv "${name}" \
      "${name/(#b)(<->)/${(l:3::0:)match}}"
done

Numbered filenames without padding

Filenames with numbers in them are not uncommon. But sorting them always has an unexpected conclusion.

file1.txt
file100.txt
file2.txt
file200.txt

This is because sorting is an operation based on individual character codes rather than words or numbers which are both strings of characters (or digits).

Pad it

The well‑known solution is to rename the files such that the numbers are all of the same length. So if the numbers go from 1 to 100, all the numbers have to be three digits long

file001.txt
file002.txt
file100.txt
file200.txt

But that’s a slow process, so obviously one has to do it faster

Pad it programmatically

NB These are instructions for zsh and most likely will not work in other shells

Hence the expanded parameter

${name/(#b)(<->)/${l:3::0:)match}}

that is broken down.

substitution

${name/pattern/replacement}

Overall, it is a substituition. ${name} is the parameter (assuming name holds a filename). zsh will look for the first occurence of pattern in name and replace it with the value of replacement.

$ name='John Doe'
$ echo "${name/o/u}"
Juhn Doe
$ echo "${name/oe/onne}"
John Donne

number pattern

The search pattern (#b)(<->) has two parts. Let’s first look at the <->.

<n-m> is a glob pattern in zsh to indicate a range of numbers— not digit characters. This is useful because the more common [0-9] pattern only matches one digit. But numbers in filenames are longer.

The pattern <-> is shorthand to mean any number.

$ name='May 1968'
$ echo "${name/<->/XXXX}"
May XXXX

Once the shell option extendedglob is enabled in zsh, the other part (#b) enables back-referencing for all subsequent patterns enclosed in parantheses. Back-referencing stores the matched pattern in a variable called match.

Hence (#b)(<->) finds a number and then stores it in a variable called match.

padded replacement

The ${(l:3::0:)match} which is used as the replacement in the main expansion is essentially ${match}—the matched number from earlier—with a qualifier enclosed in parentheses.

The (l:3::0:) qualifier to a parameter pads the paramater to 3 characters on the left with the character 0.

(l:3:) would truncate or pad a parameter as needed to make it three characters long. By default, padding is done using blank spaces, but in ::0 we specify to use zeros.

pattern‑match‑pad‑replace

Bringing it all together, we have ${name/(#b)(<->)/${(l:3::0:)name}} in a for-loop

TLS certificates are handled by GitHub Pages, or Netlify where I usually host my static sites. And for the few web services I selfhost on a cloud server, I use nginx-proxy with its companion service for requesting and installing certificates from letsencrypt.org.

I had a vague idea of how to generate self-signed certificates using openssl. But my understanding was so vague that I had to refer to some guide online each time.

And the complexity of generating self-signed certificates is complemented by the fact that they are not trusted by most clients due to the abscence of signatures from a known and trusted Certificate Authority.

And so began my journey to become my own certificate authority.

What I Already Know

HTTP

The hyper-text transfer protocol is at the application layer of communications that make up the world wide web – the web. It is built on top of TCP and IP

HTTPS

The secure version of HTTP uses transport layer security – TLS – to encrypt the data packets that are being trasmitted. This ensures that nobody other than the client and server will understand the messages being exchanged between themselves.

TLS

A layer between TCP and HTTP for encrypting HTTP packets. It is deployed by installing something to your webserver. I almost always use nginx, so to me that meant saving a certificate file, and a key file and configuring nginx webservers to use them. This meant specifying the ssl_certificate and ssl_certificat_key directives in the nginx configurations.

Generating the TLS certificate

I have had to keep referring back to online guides to get this done.

openssl req -new -x509 \
    -newkey rsa:4096 \
    -nodes -sha256 \
    -days 365 \
    -keyout key.pem \
    -out cert.pem

And this gave me two files – a certificate and a key – that I could install on my server.

Trusting on the client

Open client, normally my browser, settings and import cert.pem from the previous step. On my android phone, I install the certificate system-wide.

What I Want Now

I wanted to improve my setup in the following ways

• use elliptic curve keys; that are supposedly faster and also more secure

• be a certificate authority; to avoid trusting new certificates each time

Getting To What I need

The importance of the key

My understanding thus far of most cryptographic operations had been vague. But now, I finally realised the private key was the more basic element from which the certificate was derived.

I had come to the realise that the private key is also the encryption key. In retrospect, that is quite obvious! I just had not thought enough about what I was doing.

The private key is used to encrypt data. But first, I sign a certificate with it. Then I get it signed by an authority so my certificate is proven as trustworthy.

And there is usually a verification step to show that I do control the domain name in the certificate.

Elliptic Curve key

Trudging through the documentation, I did not really fully understand, but I found an example in the manpage openssl-genpkey(1).

openssl genpkey \
    -out privkey.pem \
    -algorithm EC \
    -pkeyopt ec_paramgen_curve:P-384 \
    -pkeyopt ec_param_enc:named_curve

Going a bit further, I added the -aes256 flag to encrypt the key output. The password was collected in the openssl prompts.

Becoming the certificate authority

Turns out becoming a certificate authority is pretty simple. I just had to create a self-signed certificate with they key I wanted to use. And then use the same certificate to sign other certificates.

openssl req -x509 \
    -key privkey.pem \
    -out CAcert.pem \
    -sha512 -days 365

openssl-req is the subcommand to generate certificate requests (more on that later), but combined with the -x509 flag, it generates self-signed certificates. I do not need to specify -new as I did before because it is implied when used with x509.

So now that I had a satisfactory certificate linked to an elliptic key, I installed it on my devices.

Certificate signing request

Until now, I had only made certificates that were signed by the key. But now that I wanted to have it signed by a certificate authority, I had to generate CSRs. These are created using the same command as self-signed certificates with the -new instead of the -x509 flag.

First I generate a new private key - using an elliptic curve! Then a CSR using the key.

openssl req -new \
    -key domainkey.pem \
    -out domain.csr \
    -sha512 -days 365

Authoritatively signed certificates

Now to use the original key privkey.pem and its certificate to sign the CSR and generate a new trusted certificate for the domain.

openssl x509 -req \
    -out domaincert.pem \
    -in domain.csr \
    -CA CAcert.pem -CAkey privkey.pem \
    -sha512 -days 365

and done!

I now have a domaincert.pem and a domainkey.pem that I can use to set up https on my self-hosted webservers. And I also have a CAcert.pem that I can install on my devices to trust the current and future domain certificates.

But not all VPN advertising is fake, and some aspects of it are rather useful—in ways that are not often portrayed by VPN providers.

Skip the ranting tangent (Not recommended)

What is a VPN?

In essence, it is exactly what it says. A private network that is virtual.

What is a network?

Connection between two more devices. In the context of the internet, the connection is usually established using

• copper cables
• radio waves
• fiber optic cables

A virtual nework?

In computing, virtual refers to hardware that does not exist physically. It is existing hardware but in software it appears as a device that is distinct from what exists physically.

A now common example is virtualised CPUs. A physical CPU that has sixteen cores can be presented in software as two CPUs with eight cores each.

Likewise, virtual networks use the existing network hardware but appear as a completely separate network in software.

So a Virutal Private Network?

To be fair, a lot of networks are private in that they cannot be accessed by members who are not directly connected to the network. Every home router of today sets up a LAN—local area network. And only the devices connected directly to the router can access resources on the network (printers, other computers, etc.)

Normally such networks are restricted and no other computer from the internet is allowed access. While it is possible to open the network to others on the internet, it poses the security risk of unauthorised access.

Instead, by using virtualised networks one can allow access to certain devices on a network while using the existing hardware of the internet. Such networks, at least the modern implementations, typically use more secure ways to communicate and authenticate devices to one another.

Consumer VPNs

XXXXX VPN encrypts and routes your network through our secure servers in N global locations…

It’s exactly what it says. Instead of joining the virtual network to access other devices on it, all connections to the internet are made through only one device of the network.

And so,

• the internet service provider sees a lot of requests to XXXXX VPN’s server
  • but not the actual site that you visit
• The site that you visit sees a lot of requests from the XXXXX VPN server
  • but not from your actual device/IP address

This functionality is proxying – communication between two devices via an intermediary called a proxy.

It could be sort of anonymous. But VPN services are obliged to log and share logs with government authorities in some countries. And if the site has a way for users to login (as do most common ones today) the site knows who it is, just not your actual location, unless it already had access to device location and does not rely on IP address for locating in the first place.

The service offerred by majority of consumer VPN providers is essentially a web proxy. While they offer partial anonymitiy, it really depends on how it is being used. It irks me that advertising today makes it look like an all in one solution.

…allowing you to access geo-restricted content…

This is possibly the only real use for most consummer VPN services. And luckily, a lot more advertising is stressing on this more than the “encryption” and “anonymity” that they offer. Hurray! 🎊

Perhaps this Tom Scott video on VPN advertising claims from 2019 shifted the industry. Notably later on in 2022, Tom Scott himself concedes that VPNs are useful in the sponsor segment of a video. And the conecession was that they are useful forrr pretending you are visiting a site from a different country. 🥁

Hah. Vindication.

Enterprise VPNs

But not all VPNs are the same. Many large organisations deploy and use their own VPNs to allow their staff access to company resources from outside the premises. Using a VPN adds a layer of protection against unauthorised access by keeping the network private.

Using VPN to Expose the Home Network

Today is the third day of writing this post, and I have entered the main topic only now. Will I ever stop wandering?

I run some network-accessible services on a Raspberry Pi 400 on the home network that acts as my “server.” It’s mostly just a Syncthing instance to handle backup of my phone’s camera folder and a Photoprism instance to view said backups.

I would expand on it some day. But for now, it’s just two services.

And sometimes, I wished I could access these from outside home: on my phone, on a 4G connection.

How does one set that up?

By hosting one’s own VPN. Which often requires having at least one device with a public IP address.

OpenVPN

I tried setting up OpenVPN first. But I could not understand any of what I was doing. I did set up a docker container based on docker.io/kylemann/openvpn. But not knowing what I was doing was troubling.

Wireguard

Wireguard was supposed to be easier to set up. It’s new and reportedly more secure out of the box.

Others

Um… I didn’t check 😅

Wiring up Wireguard

Installation

sudo apt update
sudo apt install wireguard-tools

It wasn’t too bad with Raspbian, eh? Good thing it’s based on Debian 11. I won’t be complaining for a while.

Education

Then I went to read the documentation. Okay, I did read some beforehand too. I read the quickstart guides off the official website.

I understood a little. Then I read the man pages. And I stopped trying to understand and just started smashing buttons.

Also it has been a week since I started writing this post and I don’t remember what I did.

Configuration

I started by using wg set but soon realised it wasn’t easy. I somehow figured out wg-quick.

Finally found the systemd service wg-quick@.service. So now I had an /etc/wireguard/wg0.conf file to manage my configuration.

Eventually got it setup on two devices. Using the systemd service. But what I didn’t know was that the service had a reload command. I could have used systemctl reload wg-quick@wg0. But I realised it only much later.

I used a very complicated wg syncconf <(wg-quick strip wg0) that seemed to fail at times

The configuration file itself was simple enough

[Interface]
Address = 192.168.2.2/31
PrivateKey = RandomKeyThatIsWeirdlyShort

[Peer 1]
PublicKey = AnotherKeyButPublicStyle

[Peer 2]
PublicKey = AnotherKeyButPublicStyle

[Peer n]
PublicKey = AnotherKeyButPublicStyle

I had added every single device’s public key to every device’s configuration. So every wirguard instance had the public key of all its peers.

Everything should be working fine, right?

Wires crossed

Of course not. The ‘public key’ was not enough to make a connection. Key exchanges are for authentication. Wireguard also needs an ‘endpoint.’

At first I had expected there to be some kind of global signalling server that handled this. Similar to how Syncthing establishes connections between devices.

Since that did not happen, I tried giving the private IP address assigned to each interface as the public endpoint. Of course it didn’t work. But I tried it because one of my peers had a public IP as its endpoint, so I thought if all peers connected to said public peer they should also be able to speak to each other via the public peer. And since the public peer would know the internal IPs I hoped they would suffice as endpoints.

Nope. They did not suffice. IP forwarding and things were needed. Things I did not know or understand until…

Unofficial WireGuard Documentation to Detangle

I discovered some community-created documentation for WireGuard and found out that the setup I was going for is called a “bounce server.”

To configure a WireGuard network composed of a mixture of devices with public and private IPs, one or more fo the public peers would be configured as bounce servers.

A bounce server

So far I configured peers as having an internal IP address with a 32-bit prefix. But instead a bounce server has a 24-bit (or smaller prefix)

[Interface]
Address = 192.168.2.1/24

In doing so, I let wireguard know that the current peer may receive and forward traffic for other peers within the same subnet.

A peer that handles connections to the internet would have a prefix of 0 presumably.

Connecting to a peer via a bounce server

Normally each peer needs the public key of every peer it expects to connect to. But when part of a network that involves a bounce only the bounce is configured as a peer. I expect this means the bounce server decrypts traffic from one peer, then encrypts and sends it to the next using its own keys.

I am not too concerned by this since I have complete control over my bounce server. And also because the alternatives are STUN server which are more complicated.

Bounce not forwarding

I had set everything up as I should, and every peer was connected to the bounce. But I could not connect from one peer to the next over the wirguard network.

I decided to try enabling the net.ipv4.ip_forward kernel parameter that every guide was talking about. But it was already enabled.

IRC for help

Seeing no other way to resolve and get it up and running I went to the recommended IRC room for help.

Turns out my firewall was blocking IP forwards. My server, running Fedora, used firewalld— the opposite of uncomplicated.

It has been a month and half since I started writing this post by the way. I am just going to get this over with as soon as possible.

Future me intervenes

Today is 11 January, 2023. It has been more than four months since I started this piece.

I do not even remember the exact details. I got my Wireguard VPN working. And while I did not have many uses for it at first, it has now become one of the most important (to me) internet services I manage.

I use it to provide my Nextcloud with an external storage backend via SSH over Wireguard to my home server. So I can move off old large files that I need access to, but don’t need to store on my rented VPS.

Eventually I have come to realise that I sidetracked too much, purs uing perfection and comprehensiveness that is not suited for this blog. This entry is neither bite-sized nor journalish.

As such, this ends the entry and I will move on to other ones. And try to stick to the idea of bitesyzed tech journal

The Problem with bluetooth peripherals

Bluetooth is not too bad. The connection is quite stable once connected. But connecting is not always the fun part. And sure there is probably a little bit of latency. Not one that I can perceive. I just type, point and click on links. And best of all, no wires!

The real problem for me is that bluetooth connections are made at the operating system level.

Bluetooth and the operating system

I did not actually know that bluetooth was OS-level until I came across this bit in the Arch Linux wiki (the best Linux docs, ftw) about pairing for dual boot

To pair devices on dual boot setups you need to change the pairing keys manually on your Linux install, so that they match in both systems.

Okay to be clear, the connection is done between hardware, obviously. But the pairing is handled by the operating system. And pairing keys are linked to the adapters.

A tangent explaining bluetooth

not verified

• Bluetooth is a set of rules for exchanging information between devices using radio waves
• Every bluetooth-enabled device has an “adapter”—a device to transmit and receive said radio waves
• Each adapter has a unique MAC address to identify itself to other adapters (devices)
• When two devices are being paired—connected for the first time—they exchange pairing keys
• The operating system stores said key and uses it for future connections

This is not a problem when your bluetooth headsets have only one invisible operating system; or one phone with one OS. But with a dual boot system, you need to copy the pairing keys so both OSes share the same keys since because assuming that the computer shares the bluetooth hardware, the MAC would be mapped to a specific pairing key on the other device.

GRR.

But this is easily solvable. I do it all the time on my personal laptop to connect my headphones to both Fedora and Windows.

The real problem for me

is that I could not switch between OSes without using another keyboard. I would need to press keys at the GRUB menu to change OSes, but that would not be possible with a bluetooth keyboard because connections are OS-level, not at the firmware level.

So GRUB has no way of receiving this input.

Logitech Unifying

Now I already knew both the multi-device mouse and the multi-device keyboard supported Logitech’s Unifying receiver and each came with its own tiny one. But I had never used it.

I assumed that connecting or pairing would just be plug and pair. Those are the kinds of receivers I had used so far. The device came with one dongle and worked only with that one.

But instead it wanted some software??? Oh no. I thought it meant I needed the software everywhere, and that it would immediately mean no Linux compatability and definitely not working in firmware.

StackExchange to the rescue

Google played a minor role too

Looking up “Logitech Unifying + BIOS” gave me the answer I needed. It was possible to use the the keyboard with the receiver in firmware menus!

I took a look at YouTube also because Logitech has a pretty good stock of content on their channel. And WOW those videos are OOOLD!

This has been around since 2009???

That is so cool!

Connecting via the Unifying Receiver

Turns out the special software is needed only to make the connection. The pairing process so to speak. And this time the pairing information is stored on the receiver itself; not the OS. So it becomes possible to use it on any system, it just reports itself as a regular old wireless keyboard and mouse!

I can use it in firmware, GRUB, Linux, Windows. And even on a completely different computer because now the device is paired with the receiver, not the computer.

I am disappointed

That I did not know this before.

Is a section of the official website of the Vite project. Trying to read it without understanding ESM was what led me on the previous rabbit hole, where I checked out modules

Since then, I am a bit clearer on JavaScript modules in the browser. And also no longer in the mood to read so much documentation. Time to dive in blind and figure it out.

Initialising a Vite Project

I jumped to the Getting Started section in Vite’s homepage and found the commands to create a Vite project

npm create vite@latest

or

yarn create vite

I was using a container based on the docker.io/kevinnls/node-unprivileged image I wrote (coughs—that very badly needs updates and automation) with a docker-compose spec to mount the project directory to the correct destination within the container.

So I set these files up before running yarn create vite .… and the vite-create package strongly insisted on removing everything in ..

I thought to myself that I could just restore the files using git since I had already checked them in. Then I chose a vanill + TypeScript config for the prject.

Ha. Ha. create-vite deleted the .git folder too. This is not a good start.

$ git clone <url>
$ cd <repo>
$ docker compose run --rm dev sh

node@container $ cd <mount point>
node@container $ yarn create vite
node@container $ # chose vanilla + ts
node@container $ mv vite-app/* vite-app/.* .
node@container $ rmdir vite-app
node@container $ exit

$ docker compose up -d

Well that was complex. But it did not fix everything

The container exited immediately

And to debug, I ran it again, duh. And when that did not fix it, I checked the logs.

The container’s command was yarn run dev and it could not find a binary for vite. Oh. I did not install any of the packages.

$ docker run --rm dev sh

node@container $ cd <mount point>
node@container $ yarn
node@container $ exit

$ docker compose up -d

I had already changed the publish ports to match the server in the container.

But I still could not access the server from the host.

Exposing the Vite container’s dev server

Turns out the Vite dev server exposes only to localhost (as do most others) and my command needed the --host flag

Modify the compose spec further. New command now yarn run dev --host.

The file tree

I went over the files that were created by create-vite.

• package.json

  A scripts object was added with the dev, build, preview commands and the corresponding vite commands.

• index.html

  Basic HTML that every JS framework generates. An empty div to be manipulated later by scripts. And a script that is imported as a module. I have not really noticed this before. Do all frameworks use module now?

• tsconfig.json

  TypeScript configuration options that I don’t really know anything about. The only thing that stood out to me was the include key whose value was ["src"] where the main.ts file is saved.

• src/main.ts

  It was just one very big string that got inserted into the container div. And it calls a setup function definied in another module

• src/counter.ts

  Exports the setup function I mentioned earlier that adds a click listener on a button that’s a counter

Looks like counters are the demo of every framework

• src/vite-env.d.ts

  No iead what this is. But I don’t know enough TS for this to make sense. Yet.

• a bunch of static files

  Styles, SVGs. But why are we importing styles into the modules? I don’t know. We don’t import it in the index.html file I realised later (as in, just now). I suppose that’s part of the vite magic? Applying CSS through module imports? Or did I just not realise it was possible?

And it serves TypeScript files…?

I saw the imported script file in index.html was still src/main.ts. Interesting… At the moment, the browser is not a runtime that can execute TypeScript (and I don’t think it ever will be).

Heading over to view-source://localhost:8080/src/main.ts and I read pure JavaScript. So it’s serving a file with .ts but it’s a JS file. But browsers wouldn’t know what to do with the .ts extension– oh wait. Browsers don’t care about extensions. Only media types.

$ curl localhost:8080/src/main.ts -I
...
Content-Type: application/javascript
...

And it’s the server that sets the media type. MIME type. That’s the word. I have been writing this for too long. Procrastination. And I cannot speak nerd anymore.

So what does Vite do?

It serves the project source. That’s it. Well, that’s all I know.

So far every website I have ever built was either created using a JavaScript framework’s default tools, Jekyll, or plain HTML/CSS/JS.

I decided I wanted to learn to build websites using build tools that I configured myself. Vite has been the most popular as far as I know. So I went to their homepage. The helpful ‘Why Vite?’ link caught my attention. Navigating to the next page, I soon lost attention.

I had no idea what ‘modules’ were. And that is all that the page was talking about.

What are modules?

Is the question I embarked to answer. Of course by modules, I mean ESM–ECMAScript Modules.

MDN’s article on JavaScript Modules is where I begin my journey. To play around with the concepts I started building a website that was definitely not inspired by Svelte’s tutorial.

It was simple enough. A numeric counter that updates based on user input. The user is given two buttons: one to increment the counter and the other to decrement it.

But what’s the point of having modules if there’s only one thing done?

Observing the counter

aka yak-1

To keep experimenting with modules, I did add a few console.log() statements because duh. But I also went a bit further and decided to persist the counter between sessions.

Normally, I would use localStorage.setItem() as part of the button’s click event handler. This time I decided to try something different. Hoping that doing this would not block the main thread, I used the MutationObserver API to watch the DOM change and store to localStorage when it does.

Might be better to use a promise or some mechanism to debounce. But those are things I thought of now; not when I was working on it.

A Spinner

aka yak-2

Because I was storing to localStorage, I had to read from localStorage on page loads. While the browser storage is fast, it is visible when the value changes at initial load.

So I wrote a spinner SVG. COMPLETELY BY HAND! Okay I mean manually.

I even added animation! TADA!

The inspiration was a recent episode of HTTP203 titled “Demystifying SVG Paths”

the episode worked! i wrote some SVG manually! 🐢
then i forced the user to see it because i wanted to show off

— Kevin Samuel (@kevinnlsamuel) July 26, 2022

I now know modules

aka finish shaving yak-0

I realised only while writing this post that my entire journey learning about modules was a yak shaving trip in itself.

I had started because I wanted to try out Vite…

How I open links

I almost always open links in a new tab. Be it with primary click while holding down ctrl; or as I do nowadays with a middle click emulated by the scroll wheel or a three-finger tap on the touchpad. That was a long sentence that I most certainly could have framed better.

Does everyone else open links like this?

I have no idea. I know my mother uses the secondary-click → open in new tab flow. But I have not watched a lot of other people use their computers.

Maybe I should start watching people who code stream and observe.

What do I want links to do?

When I have an <a> tag in my site, during the initial stages of making this blog I thought

I want all links to always have target=_blank

But now I am wiser. I know that

1. I want links embedded in my post to have _blank as their target
2. I want navigational links to targe _self

Today I discovered that it may be possible to achieve this using HTML!

What’s a blog post by me without a long and unnecessary tangent?

I was trying to think of a semantic tag that I could use to group a set of action buttons in my website’s header. Normally elements in a header are grouped using <nav. But I wanted to use buttons for print and contact. And neither was destined to be a navigation.

I could think of <section> but it did not seem right. Maybe I should just use a <div>. Nonetheless, I was on mdn <3 — the Mozilla Developer Network Web Docs verifying the semantics of the section tag. Then I had the idea of looking at all tags. So I went to the HTML Elements Reference

Breadcrumbs: A tangent from the tangent

The feature that many sites have started to adopt to show the logical path in the website that leads to the current page. I learnt recently it’s called “breadcrumbs.” Such a cute name. I forget where I learnt it from. This is what I get for not writing my “diary” regulary.

Also props to Google Chrome’s DevTools finally introducing node screenshots. Makes things a bit easier while writing pieces like this.

The <base> tag

While perusing the list of elements, I chanced upon the base tag. I had never heard of it before. Opening the MDN reference page for <base> I find out it’s used to specify a canonical uri that serves as the base for relative uris within the page.

But it can also be used to define the default target behaviour for anchors that do not have anything specified.

This exactly what I wanted!

But

But the target defined using base does not differentiate between internal navigation and external embeds. So everything opens in a new context :(

Maybe this isn’t the right solution. But it was a fun discovery!

Other news

Getting better at vim

I use vim as my main editor most of the time. And I recently watched a set of videos on YouTube by The Primeagen— “Vim as your editor”. And I have got three videos in and started to use some of the tricks. Slowly incorporating it into my work.

The F command has been useful. I used to keep doing f and , to go search backwards in the line to jump. Horizontal movement. It was not until recently that I even started using the f command. Used to hl like a caveperson.

Also discovered :set relativenumber from the videos. So I am also trying to use relative vertical movements instead of smashing down on jk. I even added it to my ~/.vimrc. I have not got around to using it as effectively as I would like. But baby steps, right?

GitHub Pages and Jekyll

This blog has been hosted on GitHub Pages from the very beginning. Pages is the free static-hosting offered by GitHub.

It was Pages that introduced me to Jekyll – a static site generator that runs on Ruby. And I have since had a soft spot for Jekyll. I would use it anywhere I could.

I find it easy to use and understand. The major factor for me is possibly the direct relation between source file and html. Something I had not seen in the Javascript-frameworks I had experienced until then. Abstractations frighten me. Well, some of them. I have no knowledge in Ruby after all.

Local Development

As I write the posts in watch them rendered in my browser using Jekyll’s server— jekyll serve.

But I don’t like installing development dependencies to my host operating systems so I run Jekyll in a Docker container.

Long ago, I used the jekyll/jekyll image. Took a lot of fiddling and customising to get it to work the way I wanted (cannot remember why). But I had evenually discovered GitHub published its own Docker image for GitHub Pages. This image used the correct gem (ruby module) and installed all the other dependencies too.

Just one problem. It was HUGE. Sizes beyond a gigabyte. Something I did not prefer.

They did have a Dockerfile for Alpine images. But they just never built or published it.

Local Development on a Pi

And another hitch I ran into was when I ran the server off my Rapsberry Pi, a computer running Linux on the ARM architecture.

The image – the whole gigabyte of it – downloaded without complaints. But in the end it wouldn’t run because it was not the supported architecture. Well that was disappointing.

New Images from the Gem

So I forked and cloned the entire repository for the Pages gem and image.

It had a Makefile with a defined target image_alpine

But I didn’t want to have to clone the entire repo each time on each system I needed the image. Not even a shallow clone. I really don’t like downloading development things I don’t need.

So I took it upon myself to use GitHub Actions—the CI/CD service offerred by them and build and publish the Alpine image for myself.

The original repo already had some configuration for this since that was how the images were built upstream too.

After 16 commits and lots of failed builds I finally got the images I needed building automatically on my fork using GitHub’s infrastructure.

Thoughts on GitHub

But with all this, it certainly does seem like I am locking myself to the vendor (read Microsoft). Hosting my code, my site, building my images, publishing my images. Everything on GitHub.

But I am not necessarily a fan of the platform. At least not as much as I used to be.

The recent questions of ethics that have been raised with GitHub charging for their Co-Pilot service is a minor reason for it. I also find the Actions platform rather complicated. GitLab CI was quite simpler. But I do enjoy that the yaml files are in a subdirectory and that I can have more than one, something GitLab doesn’t seem to support.

I do enjoy that they have a CLI app. But I have been using it less and less. And overall GitHub does feel less refined and more cluttered that it used to be when I started using it.

I do plan to eventually go completely self-hosted for my own repositories. But that’s a way off in the future. Until then, GitHub it is.