Moving to an eSim

With an iPhone 14 Pro on order, I’ve been reading about eSIMs. My iPhone 12 Pro has a physical SIM card, and room for an eSIM as well. The new iPhone 14 Pro no longer has a SIM card slot (at least in North America); rather than wait for the day it arrives to convert, I went ahead and did it today.

A quick search on Duck Duck Go brought me to About eSIM on iPhone at Apple Support site. Following the steps outlined in the “Convert a Physical SIM to an eSIM on the same iPhone” section was straight forward and quick to do.

Open up the Settings app, and then tap on Cellular. Tap the “Convert to eSIM” button. Wait. Once it is done, power off the phone, remove the physical SIM, and power the phone back on. Voilà.

Now, when my new phone arrives, I only have to manage the transfer from old to new.

The Last Algorithms Course You'll Need ⧁

A free algorithms course on Front End Masters. While algorithms were discussed in some of my college courses, there wasn’t a dedicated course on this subject.

Automating a Reading List

For a very long time I have been searching for, and trying, different note taking schemes in a futile effort to find a way to capture links to pages on the Internet. I wanted a way to find some previously read article or posting about “X”. Searching my browser history sometimes works, but not always. Creating a bookmark would only result in hundreds of bookmarks, and another searching problem. I view a bookmark as a way to return to a frequently visited site. For example, I have a book mark to Hacker News, but I don’t bookmark individual articles I find there.

On Hacker News recently there was a posting about tracking everything a person had read for a year. Underneath the statistics there was an automated way to, with a single click, capture a page you’ve read online. It works by using a JavaScript bookmarklet, some GitHub repository Actions, and a couple of Go language functions.

What You Need

JavaScript Bookmarklet

Copy the readingList.js bookmarklet Gist to your GitHub account. Make it a secret Gist as it will eventually have your GitHub Personal Access Token (PAT) included.

Fork the readingList repository

Fork the readingList repository created by codemicro. It was their idea, they deserve the forks. Once you have your fork, scrub through the code and replace all the references to codemicro with your GitHub ID.

Workflows

The repository contains two GitHub Action workflows in the .github/workflows directory: append and build. Find the run line in both of these and update the path to point to your fork of the repository.

Additionally, in the append.yml file there is an email address for notifications; make sure to update it to your email address.

Go

generator.go

In the generator.go file, toward the end of the code, line 194 in my fork, there is an HTML link that needs to be updated to point to your readingList repository.

manager.go

In the manager.go file you need to update the "github.com/codemicro/readingList/transport" import to reference your repository.

go.mod

Once you’ve finished updating all the code references to your ID or repository, rebuild the go.mod file. I deleted it and ran go mod init and the go mod tidy.

save.html

In the comments to the original readingList.js Gist, jamesmstone put links to his fork of the project. He added a new piece of JavaScript that allows you to use the bookmarklet without a server. In my case I’m using GitHub Pages to host the actual list page, the save.html code the jamesmstone created makes this possible.

Copy his save.html code to the .site directory in your fork of the project. On line 13 of the file, change his account name to yours.

readingList.csv

The links are all appended to the readingList.csv file in the readingList repository. Unless you want all of codemicro’s articles, edit this file and delete all the lines except for the heading line.

GitHub Pages

In your readingList repository you need to setup GitHub Pages. Click on the Settings link in the navigation bar. I set mine up to be <account>.github.io/readingList. For Source I picked “Deploy from a branch”. And for Branch I selected “gh-pages” and “/root”. I also checked the “Enforce HTTPS” option.

Create a Personal Access Token

In order for the JavaScript bookmarklet to function, it need a GitHub Personal Access Token or PAT. This article, Using GitHub Actions with Repository Dispatch Event explains the mechanism used to tie JavaScript and GitHub action together, using a PAT. Creating a personal access token describes how to create and use a PAT.

Add PAT to readingList.js

Once you have created your PAT, you need to add it to line 3 of readingList.js. It becomes the token used by the rest of the code to gain access to your repository.

Update requestURL

You need to update the requestURL value to be the path to your readingList. If you are using GitHub Pages this will look like https://zanshin.github.io/readingList/save.

Create a Bookmarklet

I used How to create a JavaScript Bookmarklet Easily to compress the JavaScript to a single line. Copy that line and make a bookmark of it in your browser(s).

Using the Bookmarklet

With the bookmarklet created you are ready to put the automation to work. Find an article you’d like to add to your list and click the read bookmark and (if you have all the pieces properly lined up) the page should briefly disappear and reappear. The JavaScript interrogates the page, collecting the URL, the title, the meta data description, etc, and triggers the GitHub Action. It then redirects to the original page. By using the Action this way, the adding of the new entry to your reading list happens asynchronously. You can go to the Actions page of your repository and see the status of the actions as they are running. Within a minute or two the readingList site should be regenerated with the new entry added.

Moving Parts

This automation has many moving parts. Two pieces of JavaScript, GitHub Actions, a GitHub PAT, and GitHub Pages. Getting all the references updated from the original project to your account, getting the path to the repository correct in the readingList.js, making sure that GitHub Pages is setup properly; all these steps are crucial. Tracking down one missing or incorrect piece (the repository path, ahem) can be frustrating.

I had never used GitHub Actions prior to this, and my understanding of JavaScript is rather thin. I spent most of a day reading and learning to make all the parts of this make sense in my head, and making them all work together to have a functional reading list.

Update

I’ve been using this automation for several days now and it works very well. So far there has only been one site where the JavaScript bookmarklet wouldn’t work. I’ve been going through the many open tabs in my browsers and adding articles to my read list.

Taz

{{ $image := .ResourceGetMatch “taz.jpg” }}

Taz (2003 - 2022)

How to Setup Lua Autocmds for Neovim

The latest nightly builds of Neovim now have Lua-based autocmd support. Previously autocmd and augroup commands needed to be wrapped inside a vim.cmd block so that the Vimscript-based statements would work.

The nightly builds now include nvim_create_augroup and nvim_create_autocmd commands that allow you to create auto command groups and auto commands without resorting to nesting Vimscript in your Lua-based configuration.

Helpers

I created two helper functions: one for creating auto groups, and one for creating auto commands.

    local agrp = vim.api.nvim_create_augroup
    local acmd = vim.api.nvim_create_autocmd

Auto groups

For each set of related auto commands I wanted, I created a group. So that the group names will sort to the top of the output when running :au <event>, I preceded each name with an underscore. For example, this group is where I keep any command that is aimed at all file types.

    local _general = agrp("_general", { clear = true })

The clear = true isn’t strictly required, as it is the default. But I like it as a reminder that the group will clear previously set commands.

Autocmds

The format of the the nvim_create_autocmd statement took me a little bit of experimenting to figure out. Here is an example of what I ended up with.

    acmd({ "FocusLost" },
         { pattern = "*",
           command = ":wa",
           group = _general })

The first dictionary defines the event (or events) this autocommand is triggered by. The second dictionary defines the pattern to match, the command to run, and identifies the group the command belongs to. In the example above, the event is FocusLost, the pattern is *, the command is :wa, and the group is _general.

I have a number of groups and commands for various file types. Even for relatively short, simple commands I followed the same multi-line format. This way all the autocmd groups in the file look the same. The file itself can be viewed here.

Converting my Neovim Configuration to Lua

Professionally I started using Vi in 1997 on AIX. Personally I started using Vim around 2008 when I began using Octopress as my static site generator. In November 2011 I started keeping the configuration in a Git repository. In December 2014 I started using Neovim in addition to Vim. Eventually my use of Vim tapered off, and for the past several years I haven’t bothered to keep my Vim configuration up-to-date. It has now been deprecated in my dotfiles repository in favor of Neovim.

The size and complexity of my configuration has ebbed and flowed over time; generally trending toward more complexity and greater size. When I set out to migrate to a Lua based configuration file, my init.vim file was just over 1000 lines long, excluding comments and white space.

Why Convert to Lua?

To paraphrase Gregory Mallory, “Because I can.”

All kidding aside I converted for two reasons. First I don’t know anything about Lua , and this was a chance to learn a little about this language. Second, it would force me to examine my entire configuration, allowing for some house cleaning.

The support for Lua is still evolving, so having your configuration written in Lua is pretty close to bleeding edge. For me, tinkering with my Neovim setup is part of the enjoyment of using Neovim, so I’m willing to endure some pain caused by being closer to the leading edge of development.

Objectives

Modular

Rather than have a single, monolithic file, containing my entire configuration, I wanted to modularize my setup. Fortunately many of the examples I found on GitHub and the r/vimporn and r/neovim Reddits are broken out into directories and files.

Concise

Only install the plugins I have a use for. My previous Neovim configuration had gotten crufty, bloated even, with plugins I no longer used, and with odd mappings I no longer remembered the purpose for.

Portable

I have several computers of my own, and a couple provided by my employer, I need this configuration to be portable and relatively easy to install in a variety of environments. My preferred OS is MacOS, but I work on Ubuntu and AmazonLinux servers professionally, and I have a Linux laptop for personal experimentation.

Process

Not wanting to corrupt my current Neovim configuration, I chose to start experimenting with a Lua configuration on a Raspberry Pi. I simply didn’t setup my configuration when I installed Neovim on the Pi. Since this meant I was using the configuration I was making to edit the configuration I was making (eating my own dog food), I was painfully aware any time I managed to break something.

When, in the course of exploring other people’s configurations, I discovered some new plugin that I wanted right away, I’d add it to the init.vim Neovim configuration on my other machines, by embedding the Lua code in the Vimscript base.

Eventually I felt I understood enough about how to structure a Lua configuration, that I started in earnest with a new branch of my dotfiles repository. That branch continued to improved and I started using it for everyday use about a month ago.

This past weekend I watched nearly all of the videos in the “Neovim from Scratch” series on YouTube, which resulted in a major refactoring of my setup. The result is cleaner and better organized. It is also more robust.

Organization

My Neovim configuration is organized into several directories and about 40 files. While this may seem like a lot, the structure is straight forward and easy to understand.

nvim
├── lua
│   ├── config
│   │   ├── lsp
│   │   │   ├── settings
│   │   │   │   ├── jsonls.lua
│   │   │   │   └── sumneko_lua.lua
│   │   │   ├── handlers.lua
│   │   │   ├── init.lua
│   │   │   └── lsp-installer.lua
│   │   ├── cmp.lua
│   │   ├── gitsigns.lua
│   │   ├── gundo.lua
│   │   ├── lualine.lua
│   │   ├── nvim-comment.lua
│   │   ├── nvim-tree.lua
│   │   ├── tabline.lua
│   │   ├── telescope.lua
│   │   ├── toggleterm.lua
│   │   ├── treesitter.lua
│   │   └── which-key.lua
│   └── usr
│       ├── autocmds.lua
│       ├── colors.lua
│       ├── helpers.lua
│       ├── mappings.lua
│       ├── options.lua
│       └── plugins.lua
├── plugin
│   └── packer_compiled.lua
├── spell
│   ├── en.utf-8.add
│   └── en.utf-8.add.spl
├── .gitignore
├── README.md
└── init.lua

nvim Directory

The nvim directory is located in ~/.config. It contains the init.lua file, a README, the Git repository and ignore file, and a lua directory. The init.lua file has a list of requires, one for each of these categories.

• autocmds
• colors
• helpers
• mappings
• options
• plugins

autocmds, colors, mappings, and options all contain what you would expect: auto commands, my color scheme, all my key mappings, and all my options.

helpers has several functions that are useful for creating mappings or setting options.

plugins sets up my plugin manager of choice, and all the plugins I use.

The actual files referenced by these requires are kept in ~/.config/nvim/lua/usr. Putting them in a folder under the lua directory creates a namespace, which is useful in avoiding collisions with files that might be included in plugins added later. The namespace directory can be called anything, I chose usr since the contents are for me, and since user might show up in a plugin. Many people use their GitHub account name for this namespace directory.

lua Directory

The lua directory has two sub-directories: config and usr.

config is where the configuration files for plugins are kept. For any plugin where there is a configuration file, that file is kept here. Since setting up and maintaining language servers is slightly different than most plugins, there is a separate directory under config for LSP specific configurations.

As described above, usr contains the files that describe my mappings, options, auto commands, color scheme, and plugins.

Specific Examples

The entire configuration can be viewed and cloned from my dotfiles repository. However there are some specific examples I think are important enough to draw attention to.

Use of protected calls

Including a plugin via the Lua require statement will result in an error if the plugin can’t be found or isn’t available. When this happens the Neovim configuration won’t load properly. Using the Lua pcall function to wrap the require allows the status of the call to be captured and tested, thus protecting the rest of the configuration process.

Each of my plugin configuration files has this code block at the start of the file.

local status_ok, plugin_handle = pcall(require, "plugin_name")
if not status_ok then
  return
end

The actual name of the plugin is substituted in for plugin_name. plugin-handle is a local variable that is used by the rest of the file as it points to the instance of the plugin returned by the require statement. A print statement or vim.notify statement could be added just ahead of the return, if you wanted to provide some visible feedback in the event of a failed require.

Packer

Automatic Install

I’m using Packer to manage my plugins. The following code block will automatically install Packer if it isn’t already installed.

local fn = vim.fn
local install_path = fn.stdpath('data')..'/site/pack/packer/start/packer.nvim'
if fn.empty(fn.glob(install_path)) > 0 then
  PACKER_BOOTSTRAP = fn.system {
    'git',
    'clone',
    '--depth',
    '1',
    'https://github.com/wbthomason/packer.nvim',
    install_path,
  }
  print "Installing Packer, close and reopen Neovim."
  vim.cmd [[packadd packer.nvim]]
end

In essence this clones the GitHub repository for Packer into the proper location in the file system.

Refresh on Save

This block of code will trigger a :PackerSync command any time the plugins.lua buffer is written. Very useful for updating the current list of plugins.

vim.cmd [[
  augroup packer_user_config
    autocmd!
    autocmd BufWritePost plugins.lua source <afile> | PackerSync
  augroup end
]]

use and get_config

All of the plugins are managed inside this code block.

return packer.startup(function(use)

-- plugins go here

if PACKER_BOOTSTRAP then
    require('packer').sync()
  end
end)

Each plugin is identified by a use statement.

use { "plugin_name" }

It is possible to specify dependencies on other plugins as a part of the use statement.

use {
  "plugin_name",
  requires { "dependency" },
}

It is also possible to specify the configuration file for the plugin here.

use {
  "plugin_name",
  config = get_config("plugin"),
}

get_config is a small helper function I included in the plugins.lua file.

local function get_config(name)
  return string.format("require(\"config/%s\")", name)
end

Plugins

Currently I am using the following plugins.

• packer - Plugin management
• nvim-cmp - Code completion
• gitsigns - Visual Git activity markers
• gundo - Wrapper around basic undo
• lualine - Status bar
• nvim-comment - Toggle comments on or off
• nvim-tree - File explorer
• tabline - Tab and buffer status line that integrates with lualine
• telescope - Fuzzy finder
• toggleterm - Wrapper around terminal display within neovim
• treesitter - Abstract symbol tree tool used for code syntax operations
• which-key -key - Popup command Completion

There are others (dependencies and ancillary plugins) I haven’t listed. See the my GitHub repository for the complete set.

Conclusion

I’ve been using my “new and improved” Neovim configuration for several days now. Other than a couple of minor tweaks to plugin settings and mappings, it has worked flawlessly. I’ve been able to install it on all my computers, with very little effort. The only part that isn’t complete is the tracking of words I’ve added to the spelling dictionary.

Neovim continues to be my favorite tool, and tinkering with its configuration is a very satisfying activity.

Appendix

These are some of the sources I used while converting from a vimscript based configuration to a Lua based configuration.

• https://oroques.dev/notes/neovim-init/
• https://github.com/rockerBOO/awesome-neovim#snippet
• https://microsoft.github.io/language-server-protocol/implementors/servers/
• https://blog.inkdrop.app/how-to-set-up-neovim-0-5-modern-plugins-lsp-treesitter-etc-542c3d9c9887
• https://github.com/ryanoasis/nerd-fonts#option-4-homebrew-fonts
• https://teukka.tech/luanvim.html
• https://crispgm.com/page/neovim-is-overpowering.html
• https://www.reddit.com/r/neovim/comments/kfxqcr/how_to_migrate_from_initvim_to_initlua/
• https://github.com/nanotee/nvim-lua-guide/
• https://vonheikemen.github.io/devlog/tools/configuring-neovim-using-lua/
• https://icyphox.sh/blog/nvim-lua/
• https://www.youtube.com/playlist?list=PLhoH5vyxr6Qq41NFL4GvhFp-WLd5xzIzZ

How to use Virtual Machines for Privileged Access

Objective

Separate all privileged access work from non-privileged access work as a security measure. Directly accessing servers or administrative web pages from the same machine that you read email or do web browsing is a potential security risk. To mitigate this risk, create a “privileged access workstation”, or PAW, to support all work that needs to be conducted securely.

Constraints

Separate Privileged Access Workstation (PAW)

There needs to be separation between the machine hosting the PAW and the machine used for email and normal browsing. In other words, running a VM to act at the PAW on a computer that is used for email defeats the purpose. Therefore the PAW needs to be distant from the computer used for email, etc.

In our AWS Service Catalog we have a product that creates a Linux workstation that can be access via ssh or remote desktop, provided the connection is through the VPN. The VM is only addressable via the private network; it has no public IP address. This EC2 instance satisfies the separation requirement.

Virtual Private Network (VPN)

Access to any device on the internal network requires the use of a VPN. The VPN required is Global Protect from Palo Alto. It is configured as a full tunnel VPN, so all traffic from the device connected will flow through the VPN. In practice this has caused periodic issues with applications like Zoom, Office 365, and has resulted in some performance degradation.

It is possible to write a script that splits the tunnel, routing only work related traffic through the VPN and all other traffic through the gateway address the host machine has. This script is fragile, however, as Global Protect, openconnect, and even the local operating system, all can and will have updates that break the script. It is a “high cost” solution that is outside of any support provided by the security team or the local help desk. Using the native client for GP is the least objectionable option.

Hardware

While using two separate computers, one for secure activities and one for non-secure activities would be a potential solution, space constraints, not to mention maintaining two separate physical computers, makes this solution less desirable. Instead, use a single computer capable of running a virtual machine.

Solution

I’m opting to use two virtual machines, one locally hosted on my desktop and the other hosted at AWS. The local VM will have the Global Protect native client installed, and will be used solely to establish the VPN connection. ssh and RDP traffic will flow through this machine, to and from the AWS VM.

Hosting a local VM for the VPN isolates it from any local activities, such as Zoom and Office 365. This local VM isn’t resource intensive, as it won’t be doing any work; it’s a pass through to the AWS VM.

The remote VM, hosted at AWS, is where all the privileged work will occur. It has a private network address and therefore doesn’t require the VPN for any of the activities it will perform.

The final topography is a local desktop, running MacOS 12, that has an Ubuntu 20.04 VM running in VirtualBox. The Ubuntu VM has the Global Protect client installed. At the start of the day this client is used to establish a VPN connection, which has a 12 hour time limit. Through a ssh connection using port forwarding, both ssh and RDP traffic are directed through the local VM to the AWS VM.

Setup and Configuration

Local VM

I installed the latest version of VirtualBox and downloaded an Ubuntu 20.04 Desktop ISO. I setup the VM with 4 GB of RAM and 30 GB of storage. I created a user account with the same name as my work account, and I generated an ssh keypair.

ssh-keygen -t ed25519 -o -a 100 -f ~/.ssh/id_ed25519 -C "email@example.com"

Next I installed the Linux Global Protect client. With it installed I can establish a VPN session.

Remote VM

We created a Service Catalog entry that creates a Linux administrator workstation. It is hardened to some extent, does not allow ssh via password, and creates a key pair. This workstation can be configured with all of my tools and settings.

Local Host

From my desktop I can now ssh to the local Ubuntu VM, and then ssh to the AWS workstation. It is possible to chain the two ssh commands together into one command using the -t flag. If the local VM is called foo and the AWS VM is called bar the ssh command to sign in looks like this.

ssh -t foo \ ssh -t bar

Remote Desktop (RDP) works over port 3389. Adding some port forwarding to the ssh command it is possible to pipe the RDP traffic from the local host to the AWS VM. The new command now looks like this.

ssh -t -L 3389:localhost:3389 foo \ ssh -t -L 3389:localhost:3389 bar

This command can be shortened by using the AWS VM fully qualified domain name in place of localhost in the first half of the command.

ssh -t -L 3389:bar.aws.tld:3389 foo \ ssh -t bar

In my RDP client (Microsoft Remote Desktop) I created an entry for localhost. Once the ssh command has been issued, I can open an RDP session from my local machine that resolves on the AWS VM.

Daily Usage

My daily usage pattern follows these steps.

1. Sign into the local Ubuntu VM
2. Establish a Global Protect VPN session using the native client
3. From a terminal on the local host run the ssh command to forward port :3389 to the AWS VM
4. From a terminal on the local host establish any additional ssh session I desire, leaving off the port forwarding.

I’ve been using this setup for several days now, and it is working smoothly. By keeping the resolution on the RDP session relatively small, 1920x1080, there isn’t much lag while using it. I have a tmux session on the AWS VM that I connect to once I’m signed in, so that my session is always there.

Any privileged activity that requires a command line happens in a terminal on my local machine, that is connected via ssh through the local VM to the AWS VM. Any privileged web-based activity happens in a browser running on the AWS VM, accessed via RDP.

Summary

While it would be possible to eliminate the local VM, by running the VPN on the local desktop, separating these two concerns provides enough benefit to make the extra setup and configuration worthwhile. I have duplicated this setup on my work laptop, so I can connect in the same manner when I’m using it. Thanks to tmux I’m able to pick up the same session without any effort.

I may explore the command line Global Protect client to see if it is easier to use than the GUI one. I would still have to go to the local VM to establish the VPN session, but I could do it from the command line rather than through a GUI.

Writing an OS in Rust ⧁

Continuing the “write your own operating system” theme, here’s a series of articles talking about doing that with the Rush programming language.

Linux From Scratch ⧁

Something I want to do someday, because why not?

How to Write Idempotent Bash Scripts ⧁

Tips on writing a bash shell script that will not have side effects if run more than one time.