Professionally I started using Vi in 1997 on AIX. Personally I started using Vim around 2008 when I began using Octopress as my static site generator. In November 2011 I started keeping the configuration in a Git repository. In December 2014 I started using Neovim in addition to Vim. Eventually my use of Vim tapered off, and for the past several years I haven’t bothered to keep my Vim configuration up-to-date. It has now been deprecated in my dotfiles repository in favor of Neovim.
The size and complexity of my configuration has ebbed and flowed over time; generally trending
toward more complexity and greater size. When I set out to migrate to a Lua based configuration
file, my init.vim file was just over 1000 lines long, excluding comments and white space.
To paraphrase Gregory Mallory, “Because I can.”
All kidding aside I converted for two reasons. First I don’t know anything about Lua , and this was a chance to learn a little about this language. Second, it would force me to examine my entire configuration, allowing for some house cleaning.
The support for Lua is still evolving, so having your configuration written in Lua is pretty close to bleeding edge. For me, tinkering with my Neovim setup is part of the enjoyment of using Neovim, so I’m willing to endure some pain caused by being closer to the leading edge of development.
Rather than have a single, monolithic file, containing my entire configuration, I wanted to modularize my setup. Fortunately many of the examples I found on GitHub and the r/vimporn and r/neovim Reddits are broken out into directories and files.
Only install the plugins I have a use for. My previous Neovim configuration had gotten crufty, bloated even, with plugins I no longer used, and with odd mappings I no longer remembered the purpose for.
I have several computers of my own, and a couple provided by my employer, I need this configuration to be portable and relatively easy to install in a variety of environments. My preferred OS is MacOS, but I work on Ubuntu and AmazonLinux servers professionally, and I have a Linux laptop for personal experimentation.
Not wanting to corrupt my current Neovim configuration, I chose to start experimenting with a Lua configuration on a Raspberry Pi. I simply didn’t setup my configuration when I installed Neovim on the Pi. Since this meant I was using the configuration I was making to edit the configuration I was making (eating my own dog food), I was painfully aware any time I managed to break something.
When, in the course of exploring other people’s configurations, I discovered some new plugin that I
wanted right away, I’d add it to the init.vim Neovim
configuration on my other machines, by embedding the Lua code in the Vimscript base.
Eventually I felt I understood enough about how to structure a Lua configuration, that I started in earnest with a new branch of my dotfiles repository. That branch continued to improved and I started using it for everyday use about a month ago.
This past weekend I watched nearly all of the videos in the “Neovim from Scratch” series on YouTube, which resulted in a major refactoring of my setup. The result is cleaner and better organized. It is also more robust.
My Neovim configuration is organized into several directories and about 40 files. While this may seem like a lot, the structure is straight forward and easy to understand.
nvim
├── lua
│ ├── config
│ │ ├── lsp
│ │ │ ├── settings
│ │ │ │ ├── jsonls.lua
│ │ │ │ └── sumneko_lua.lua
│ │ │ ├── handlers.lua
│ │ │ ├── init.lua
│ │ │ └── lsp-installer.lua
│ │ ├── cmp.lua
│ │ ├── gitsigns.lua
│ │ ├── gundo.lua
│ │ ├── lualine.lua
│ │ ├── nvim-comment.lua
│ │ ├── nvim-tree.lua
│ │ ├── tabline.lua
│ │ ├── telescope.lua
│ │ ├── toggleterm.lua
│ │ ├── treesitter.lua
│ │ └── which-key.lua
│ └── usr
│ ├── autocmds.lua
│ ├── colors.lua
│ ├── helpers.lua
│ ├── mappings.lua
│ ├── options.lua
│ └── plugins.lua
├── plugin
│ └── packer_compiled.lua
├── spell
│ ├── en.utf-8.add
│ └── en.utf-8.add.spl
├── .gitignore
├── README.md
└── init.luaThe nvim directory is located in ~/.config. It contains the init.lua file, a README, the Git
repository and ignore file, and a lua directory. The init.lua file has a list of requires, one
for each of these categories.
autocmds, colors, mappings, and options all contain what you would expect: auto commands, my
color scheme, all my key mappings, and all my options.
helpers has several functions that are useful for creating mappings or setting options.
plugins sets up my plugin manager of choice, and all the plugins I use.
The actual files referenced by these requires are kept in ~/.config/nvim/lua/usr. Putting them in a folder
under the lua directory creates a namespace, which is useful in avoiding collisions with files that
might be included in plugins added later. The namespace directory can be called anything, I chose
usr since the contents are for me, and since user might show up in a plugin. Many people use
their GitHub account name for this namespace directory.
The lua directory has two sub-directories: config and usr.
config is where the configuration files for plugins are kept. For any plugin where there is a
configuration file, that file is kept here. Since setting up and maintaining language servers is
slightly different than most plugins, there is a separate directory under config for LSP specific
configurations.
As described above, usr contains the files that describe my mappings, options, auto commands,
color scheme, and plugins.
The entire configuration can be viewed and cloned from my dotfiles repository. However there are some specific examples I think are important enough to draw attention to.
Including a plugin via the Lua require statement will result in an error if the plugin can’t be
found or isn’t available. When this happens the Neovim configuration won’t load properly. Using the Lua
pcall function to wrap the require allows the status of the call to be captured and tested, thus
protecting the rest of the configuration process.
Each of my plugin configuration files has this code block at the start of the file.
local status_ok, plugin_handle = pcall(require, "plugin_name")
if not status_ok then
return
endThe actual name of the plugin is substituted in for plugin_name. plugin-handle is a
local variable that is used by the rest of the file as it points to the instance of the plugin
returned by the require statement. A print statement or vim.notify statement could be added just
ahead of the return, if you wanted to provide some visible feedback in the event of a failed
require.
I’m using Packer to manage my plugins. The following code block will automatically install Packer if it isn’t already installed.
local fn = vim.fn
local install_path = fn.stdpath('data')..'/site/pack/packer/start/packer.nvim'
if fn.empty(fn.glob(install_path)) > 0 then
PACKER_BOOTSTRAP = fn.system {
'git',
'clone',
'--depth',
'1',
'https://github.com/wbthomason/packer.nvim',
install_path,
}
print "Installing Packer, close and reopen Neovim."
vim.cmd [[packadd packer.nvim]]
endIn essence this clones the GitHub repository for Packer into the proper location in the file system.
This block of code will trigger a :PackerSync command any time the plugins.lua buffer is
written. Very useful for updating the current list of plugins.
vim.cmd [[
augroup packer_user_config
autocmd!
autocmd BufWritePost plugins.lua source <afile> | PackerSync
augroup end
]]use and get_configAll of the plugins are managed inside this code block.
return packer.startup(function(use)
-- plugins go here
if PACKER_BOOTSTRAP then
require('packer').sync()
end
end)Each plugin is identified by a use statement.
use { "plugin_name" }It is possible to specify dependencies on other plugins as a part of the use statement.
use {
"plugin_name",
requires { "dependency" },
}It is also possible to specify the configuration file for the plugin here.
use {
"plugin_name",
config = get_config("plugin"),
}get_config is a small helper function I included in the plugins.lua file.
local function get_config(name)
return string.format("require(\"config/%s\")", name)
endCurrently I am using the following plugins.
There are others (dependencies and ancillary plugins) I haven’t listed. See the my GitHub repository for the complete set.
I’ve been using my “new and improved” Neovim configuration for several days now. Other than a couple of minor tweaks to plugin settings and mappings, it has worked flawlessly. I’ve been able to install it on all my computers, with very little effort. The only part that isn’t complete is the tracking of words I’ve added to the spelling dictionary.
Neovim continues to be my favorite tool, and tinkering with its configuration is a very satisfying activity.
These are some of the sources I used while converting from a vimscript based configuration to a Lua based configuration.
Separate all privileged access work from non-privileged access work as a security measure. Directly accessing servers or administrative web pages from the same machine that you read email or do web browsing is a potential security risk. To mitigate this risk, create a “privileged access workstation”, or PAW, to support all work that needs to be conducted securely.
There needs to be separation between the machine hosting the PAW and the machine used for email and normal browsing. In other words, running a VM to act at the PAW on a computer that is used for email defeats the purpose. Therefore the PAW needs to be distant from the computer used for email, etc.
In our AWS Service Catalog we have a product that creates a Linux workstation that can be access via ssh or remote desktop, provided the connection is through the VPN. The VM is only addressable via the private network; it has no public IP address. This EC2 instance satisfies the separation requirement.
Access to any device on the internal network requires the use of a VPN. The VPN required is Global Protect from Palo Alto. It is configured as a full tunnel VPN, so all traffic from the device connected will flow through the VPN. In practice this has caused periodic issues with applications like Zoom, Office 365, and has resulted in some performance degradation.
It is possible to write a script that splits the tunnel, routing only work related traffic through the VPN and all other traffic through the gateway address the host machine has. This script is fragile, however, as Global Protect, openconnect, and even the local operating system, all can and will have updates that break the script. It is a “high cost” solution that is outside of any support provided by the security team or the local help desk. Using the native client for GP is the least objectionable option.
While using two separate computers, one for secure activities and one for non-secure activities would be a potential solution, space constraints, not to mention maintaining two separate physical computers, makes this solution less desirable. Instead, use a single computer capable of running a virtual machine.
I’m opting to use two virtual machines, one locally hosted on my desktop and the other hosted at AWS. The local VM will have the Global Protect native client installed, and will be used solely to establish the VPN connection. ssh and RDP traffic will flow through this machine, to and from the AWS VM.
Hosting a local VM for the VPN isolates it from any local activities, such as Zoom and Office 365. This local VM isn’t resource intensive, as it won’t be doing any work; it’s a pass through to the AWS VM.
The remote VM, hosted at AWS, is where all the privileged work will occur. It has a private network address and therefore doesn’t require the VPN for any of the activities it will perform.
The final topography is a local desktop, running MacOS 12, that has an Ubuntu 20.04 VM running in VirtualBox. The Ubuntu VM has the Global Protect client installed. At the start of the day this client is used to establish a VPN connection, which has a 12 hour time limit. Through a ssh connection using port forwarding, both ssh and RDP traffic are directed through the local VM to the AWS VM.
I installed the latest version of VirtualBox and downloaded an Ubuntu 20.04 Desktop ISO. I setup the VM with 4 GB of RAM and 30 GB of storage. I created a user account with the same name as my work account, and I generated an ssh keypair.
ssh-keygen -t ed25519 -o -a 100 -f ~/.ssh/id_ed25519 -C "email@example.com"
Next I installed the Linux Global Protect client. With it installed I can establish a VPN session.
We created a Service Catalog entry that creates a Linux administrator workstation. It is hardened to some extent, does not allow ssh via password, and creates a key pair. This workstation can be configured with all of my tools and settings.
From my desktop I can now ssh to the local Ubuntu VM, and then ssh to the AWS workstation. It is
possible to chain the two ssh commands together into one command using the -t flag. If the local
VM is called foo and the AWS VM is called bar the ssh command to sign in looks like this.
ssh -t foo \ ssh -t bar
Remote Desktop (RDP) works over port 3389. Adding some port forwarding to the ssh command it is possible to pipe the RDP traffic from the local host to the AWS VM. The new command now looks like this.
ssh -t -L 3389:localhost:3389 foo \ ssh -t -L 3389:localhost:3389 bar
This command can be shortened by using the AWS VM fully qualified domain name in place of
localhost in the first half of the command.
ssh -t -L 3389:bar.aws.tld:3389 foo \ ssh -t bar
In my RDP client (Microsoft Remote Desktop) I created an entry for localhost. Once the ssh command
has been issued, I can open an RDP session from my local machine that resolves on the AWS VM.
My daily usage pattern follows these steps.
:3389 to the AWS VMI’ve been using this setup for several days now, and it is working smoothly. By keeping the resolution on the RDP session relatively small, 1920x1080, there isn’t much lag while using it. I have a tmux session on the AWS VM that I connect to once I’m signed in, so that my session is always there.
Any privileged activity that requires a command line happens in a terminal on my local machine, that is connected via ssh through the local VM to the AWS VM. Any privileged web-based activity happens in a browser running on the AWS VM, accessed via RDP.
While it would be possible to eliminate the local VM, by running the VPN on the local desktop, separating these two concerns provides enough benefit to make the extra setup and configuration worthwhile. I have duplicated this setup on my work laptop, so I can connect in the same manner when I’m using it. Thanks to tmux I’m able to pick up the same session without any effort.
I may explore the command line Global Protect client to see if it is easier to use than the GUI one. I would still have to go to the local VM to establish the VPN session, but I could do it from the command line rather than through a GUI.
Continuing the “write your own operating system” theme, here’s a series of articles talking about doing that with the Rush programming language.
Something I want to do someday, because why not?
Tips on writing a bash shell script that will not have side effects if run more than one time.
A fascinating dive into the world of blockchain and cryprocurrency.
A nice explanation and visualization of the graphite scale used to categorize pencils.
While setting up my new computer, one of the tasks I was faced with was recreating my Jekyll environment. Jekyll is the Ruby based static site generator I use for this site. MacOS has not always kept up with Ruby, and so I’ve had to employ rbenv to have an up to date Ruby installed. And to segregate the Gem dependencies Jekyll has from any other Ruby project that might have conflicting dependencies.
I kept putting off the task as I wasn’t looking forward to the possibility of some problems in Ruby Gem land. And then I thought about Docker—how hard would it be to use Docker as my Jekyll environment? As it turns out, not very hard at all. Eight lines of YAML total, not very hard.
A quick search lead me to this article, How to run Jekyll locally with Docker and Docker
Compose. Using docker-compose and a Docker image that has Jekyll installed on it, you can
quickly spin up a fully functioning Jekyll environment.
By putting the docker-compose.yml file in the root directory of my blog, and running
docker-compose up
I get Jekyll without having to install Jekyll, manage Gems, or use rbenv to control which version of
Ruby and which collection of Gems is in use. Best of all, since Ruby is installed by default
(version 2.6.3p62 as of this writing), I have rake at my disposal, so the Rakefile I have still
works. This file has tasks for creating new postings, or new draft postings, for publishing drafts,
and for deploying my site using rsync.
The only changes I made to the docker-compose.yml file described in the article, is that I used
different flags on the jekyll serve command. I’m using
jekyll serve --watch --drafts
which is what I’ve had for a long time. The performance of the Docker image is a bit slower than it
would be if it were natively installed. I may try --incremental at some point to see if that
speeds things up.
Here’s my docker-compose.yml file.
services:
jekyll:
image: jekyll/jekyll:latest
command: jekyll serve --watch --drafts
ports:
- 4000:4000
volumes:
- .:/srv/jekyll
Briefly, this file says, compose a Docker image using the latest jekyll base image. Once it’s ready
run the Jekyll server, watching for new drafts, and expose the server on port 4000. Finally, map the
current directory ., to /srv/jekyll in the image. Done.
Docker and docker-compose saved me a lot of time and effort. Now I wonder where else I could use it.
Recently I purchased a 24" M1 iMac. One of my goals was to have a central place for all my digital content: movies, audiobooks, music, and photos. My venerable late-2013 MacBook Pro has a 1TB drive that cannot hold my entire collection along with my other files, applications, and the operating system.
For some time now I’ve been using an external USB drive to house my media. This drive has been plugged into an even more venerable 2009 MacBook Pro. The reason for using the much older laptop as host for my media is simple: that laptop doesn’t get moved. If the external drive were attached to my primary machine, I would constantly be moving it, with the laptop, or mounting and dismounting it.
With all my music, movies, and audiobooks offloaded to the external drive, there was enough room left on my primary laptop for all my photos, and the rest of my files. It wasn’t ideal, but it worked.
The new iMac has 256GB internal storage, and a pair of 4TB external drives. I have spent
considerable time this past week consolidating all my digital media to one of the 4TB drives. I’ve
also put my software projects directory, called code, there. When I am away from my desk and want
to access something on the external drive on the iMac, those files are just an ssh connection away.
From the MacBook Pro I am using iTerm2 and ssh. From the iPadPro I’m using blink since it includes mosh, or mobile shell. The advantage to mosh is that the session stays in sync even if the connection is interrupted. Since iPadOS is fairly aggressive about stopping idle processes, using an ssh connection becomes tiresome as you are constantly having to reestablish it. With mosh, once you are connected you stay connected until you end the session.
I believe, though I am no longer certain, that the first remote session on my iMac was done from the
iPadPro using mosh. On the iMac there is a component of mosh, mosh-server the runs and manages the
incoming connection. This will play a key part in what happened with my ability to remotely access
files over ssh/mosh.
MacOS allows you to control how your computer is shared. Under the Sharing preference pane in System
Preferences, there is a Remote Login option. With that selected you can login to the local
computer from a remote computer. Once Remote Login is enabled, there is a second option, Allow full disk access for remote users. This should be enough to permit ssh or mosh session access to any of
the disks mounted.
In my case I was getting an Operation not permitted error any time I tried to access any external
drive attached to the iMac. I got this error whether I used ssh or mosh, and from any remote device:
iPadPro or MacBook Pro.
When the first remote login occurs a setting is added to Full Disk Access under the Security &
Privacy Privacy pane. This setting is called sshd-keygen-wrapper. If it isn’t listed, or is listed
and enabled, then full disk access is granted. If it is there and no enabled, however, full disk
access is not granted. In my case, sshd-keygen-wrapper was present but not enabled.
I suspect that mosh (and mosh-server) having been the mechanism for the initial remote session, left the setting in a state where no remote access to external drives was allowed. Once I enabled that setting I was able to remotely access external drives using either ssh or mosh.
This Apple discussion thread, and the links it contains, do a better job of explaining the ins and
outs of sshd-keygen-wrapper.
https://discussions.apple.com/thread/250352651 I
found this link through the ##apple IRC channel.
In the process of learning Go language programming I have consumed a lot of written and visual material. Today I listened to a few minutes of the Go Time podcast, where they were talking about documentation. The introduction to the episode really resonated with me, and so I wanted to capture it in my words here.
When first starting a new topic, particularly a technical one like, say Go programming, tutorials are a common starting point. Explanation mixed with samples, something you can follow along with at home. Learning my mimicking.
How to resources are aimed at solving a particular problem. How to I set up an HTTP router in Go?, or How should a go project be structured? A how to is more narrowly focused. It aims to impart knowledge about this one specific topic.
Somewhat broader in scope than a how to, an article or presentation (i.e., at a conference) dive a bit deeper into a topic. It isn’t a tutorial, assumptions are usually made about what you already know. And it isn’t a focused as a how to. It adds some though process to the solution or technique discussed.
Documentation is a dictionary. An reference. You are looking up the specifics of an API or library. Depending on the maturity of the documentation, and whether it includes examples or not, finding information in the documentation can be difficult. If you know the package or library you need and only need to refresh your understanding of it’s parameters and return value, documentation is fantastic. If you are unsure, and are hunting for the right tool, documentation can be frustrating.
In the course of learning a new programming language, I move through all of these categories of documentation. While working through a tutorial I may digress into a how to or look up things in the documentation. Generally though, I move from tutorials through how to and articles, and finally to documentation.