Migrating From Apache2 To Caddy

 I have wanted to investigate the feasibility of moving to a more modern webserver setup on my home server, specifically Caddy or Nginx as they both sounded like they are much easier to set up and more streamlined than apache; but never seemed to get around to it. Well, finally I manged to over this past weekend.

The main case, serving out static content from one or more domains with HTTPS cert setup, was surprisingly easy and I was impressed by its ease of use. Caddy has a nice config syntax which is easy to read, reasonably intuitive and supports single- or multi-file configurations, with a fragment syntax and include mechanism to keep large, complex configs more manageable.

In order to fully replace Apache on my home server, I needed to ensure any replacement would support a few things over and above serving just static content from a single domain:

• Multiple domain hosting (virtual hosts)
• UNIX ~username webspaces
• Use traditional syslog format instead of JSON logs
• PHP (for Nextcloud)
• CGI (Common Gateway Interface)
• Golang vanity URLs to serve out go modules

Hosting Multiple Domains (virtual hosts) with Caddy

Caddy's docs are pretty good (especially the Common Patterns section) but it's a bit tricky sometimes to know just which directive to use, where.

• Setting up multiple domain serving was very straightforward -- basic Caddy syntax and documentation spells it out clearly, refer to the Common Patterns section of the Caddy docs.
  
  

UNIX ~username webspaces with Caddy

• Hosting UNIX ~username style webspaces was pretty simple, just a basic use of the handle_path and root directives:
  
  
handle_path /~joebloggs* {
    root * /home/joebloggs/public_html
}

Use Traditional syslog Format with Caddy instead of JSON Logs

• This one was simple, just add the following entry in the log {..} section of Caddyfile config:

format transform "{common_log}"

Hosting PHP and Nextcloud from Caddy

• Hosting a PHP app behind Caddy was trickier: I couldn't find examples that spelled out precisely the proper setup. I had to first discover what the fpm-php helper util/pkg was, and how to set it up, and that it has to run with the same user:group as Caddy, and the PHP files and data that are being served (by default fpm-php runs as nobody:nogroup); simply /etc/php/fpm-php[x.y]/fpm.d/www.conf to set user=caddy and group=caddy.
• Then, finally, the Caddyfile entry for one's domain must include the proper root and php_fastcgi proxy config:

blitter.com {

        # dir as root of the website

        root * /var/www/www.blitter.com/htdocs

        # enable serving static files from root dir

        encode

        file_server

        php_fastcgi localhost:9000

} 

Hosting CGI from ~user web spaces with Caddy

• CGI was not too hard though it isn't spelled out in the Caddy Common Patterns documentation:
  
  

handle_path /~russtopia* {

    root * /home/russtopia/public_html

handle_path /cgi-bin/* {

    reverse_proxy localhost:10000 {

        transport fastcgi {

            env SCRIPT_FILENAME /home/russtopia/public_html/cgi-bin/{path}

        }

    }

}

An entry such as the above must be created for each unique user or app's local cgi-bin/ area. If CGI files must write to the user/app's backend filesystem, the intended output folder should have proper permissions to allow Caddy, if not running as root (and you don't need to run Caddy as root, so avoid it!) to access them. I created a new distinct 'cgi' group and made sure my user and caddy itself were members, and set user/group read/write perms on the cgi script's output folder.

Serving Golang Vanity URLs with Caddy

• Golang 'vanity URLs' are a way to self-host one's own Go source package modules, or redirect domain assignments such that they are referred to just as modules on the big centralized source hosts like github.com, gitlab.io etc. For example, I have authored some Go programs and modules which I make available via the root name blitter.com/go/<module>. I want the Go tooling to be able to use go get, go install, and so on without issue.
  
  I also have a rather unique git setup: I run git-daemon with packages visible from the server's /var/git/ directory, of which many are symlinked to the repo's actual location within a Gogs installation which lives and runs under the git:git user/group. In this way I can do raw git checkouts via the git:// or ssh:// schemes, as well as via https:// to my Gogs instance.
  
  This setup in Caddy was not fully covered by any other online tutorial, at least not in my server's particular setup -- I use a /go/ URL endpoint within the overall blitter.com domain to host modules, rather than dedicating a go.blitter.com domain just for Go modules, so other tutorials didn't quite fit. So here's the entire setup that integrates the git-backend as well as golang vanity URL setup:

# Caddy tutorial on serving vanity go module URLs:

# https://abhijithota.com/posts/golang-vanity-urls-using-caddy/

####

(gomodhandler) {

        handle /go/{args[0]} {

                @from_go query go-get=1

                handle @from_go {

                        header Content-Type text/html

                        respond <<HTML

                        <!DOCTYPE html>

                        <html>

                        <head>

                        <meta name="go-import" content="blitter.com/go/{args[0]} git https://blitter.com/git/{args[0]}">

                        <!-- <meta http-equiv="refresh" content="0; url=https://blitter.com/git/{args[0]}" /> -->

                        </head>

                        </html>

                        HTML 200

                }

        }

}

####

www.blitter.com {

        redir https://blitter.com{uri}

}

blitter.com {

        ####> git-daemon ####

        # Caddy tutorial on git over HTTP(s) proxying git-daemon:

        # https://www.jamesatkins.com/posts/git-over-http-with-caddy/

        handle_path /git* {

                root * /var/git

        }

        handle_path /var/git* {

                root * /git

        }

        @git_cgi path_regexp "^.*/(HEAD|info/refs|objects/info/[^/]+|git-upload-pack)$"

        @git_static path_regexp "^.*/objects/([0-9a-f]{2}/[0-9a-f]{38}|pack/pack-[0-9a-f]{40}\.(pack|idx))$"

        handle @git_cgi {

                reverse_proxy unix//run/git-cgi.socket {

                        transport fastcgi {

                                #env SCRIPT_FILENAME ${pkgs.git}/libexec/git-core/git-http-backend

                                env SCRIPT_FILENAME /usr/libexec/git-core/git-http-backend

                                env GIT_HTTP_EXPORT_ALL 1

                                env GIT_PROJECT_ROOT /var/git

                        }

                }

        }

        handle @git_static {

                file_server {

                        root /var/git

                }

        }

        ####< git-daemon ####

        import gomodhandler bacillus

        import gomodhandler brevity

        import gomodhandler chacha20

        import gomodhandler cryptmt

        import gomodhandler go-frodokem

        import gomodhandler goutmp

        import gomodhandler groestl

        import gomodhandler herradurakex

        import gomodhandler hkexsh

        import gomodhandler hopscotch

        import gomodhandler kyber

        import gomodhandler lpasswd

        import gomodhandler moonphase

        import gomodhandler mtwist

        import gomodhandler newhope

        import gomodhandler xs

        import gomodhandler xsd

}

Footnote: If switching on-the-fly to Caddy from Apache and vice-versa while testing the overall setup while an active Nextcloud instance is running involves also changing ownership of: nextcloud install and config dirs in /var/www/...; the nextcloud /data dir; *and* any existing session files, /tmp/sess_*).

Footnote 2: Sometime recently (as of go v1.2x) the go get system seemed to start requiring any go module have at least a latest tag applied otherwise the module will not be found. I had a few modules that lacked tags, so that was a real head-scratcher. Also do a go clean -modcache from time to time to ensure go is really fetching things as opposed to using cached copies, so if dependencies have broken it will be revealed when attempting go mod init && go mod tidy.

An APL Translation of 'Square Joy: Trapped Rainwater' J Posting

 I saw an interesting post entitled "Square Joy: Trapped Rainwater" on the mmapped blog, describing how to approach the problem of computing water levels in a 2D Flatland cityscape.

The configuration of bars with heights 0, 1, 0, 2, 1, 0, 1, 3, 2, 1, 2, 1, and the water trapped by this configuration.

The author uses J, so I thought I'd take a quick look at converting the basic computation into its parent language, APL. Turns out it's quite simple (minus the graphic representation which I haven't yet looked at doing in GNU APL).

J has the  /\. adverb ('suffix'), here applied to the 'insert' (/) which it shares with APL. APL must simulate it via two applications of the reversal (⌽) function:

      H ← 0 1 0 2 1 0 1 3 2 1 2 1
      ⌈\H
0 1 1 2 2 2 2 3 3 3 3 3
      ⌽⌈\⌽H    ⍝ Note the use of 'reversal' here (⌽) with insert (/) to match J's 'suffix scan'
3 3 3 3 3 3 3 3 2 2 2 1
      (⌈\H) ⌊ (⌽⌈\⌽H)
0 1 1 2 2 2 2 3 2 2 2 1

Maybe later I'll try to create the visual renderings in GNU APL to match what the J solution does.

String Interpolation in APL -- Formatted string output ala printf()

This is not as sophisticated as C's printf() of course, but it's enough for many uses.

  ∇r ← s sInterp sv
⍝⍝ Interpolate items in sv into s (string field substitution)
⍝ s: string - format string, '∆' used for interpolation points
⍝ sv: vector - vector of items to interpolate into s
⍝ r: interpolated string
  s[('∆'=s)/⍳⍴s] ← ⊃¨(⍕¨sv)
  r ← ∊s
∇
      'Mary had a ∆ lamb, its fleece was ∆ as ∆.' sInterp 'little' 'black' 'night'
Mary had a little lamb, its fleece was black as night.
      'Mary had a ∆ lamb, its fleece was ∆ as ∆.' sInterp 'little' 'large' 42
Mary had a little lamb, its fleece was large as 42.

A Recursive Depth-First Maze Generator in GNU APL

Rosettacode.org is a great place to grab little ideas and apply them to learning a new language; especially if there isn't a solution there yet in the language you're learning! This past week I took some spare time in the evenings to implement the classic maze generator problem, in GNU APL.

Takeaways:

• Sometimes just manipulating the string representation of a maze is easier than trying to come up with a clever binary representation (I started with the idea that each maze cell's walls could be a 4-bit field, eg. nsew = [3210], but the shared walls between cells made it too messy);
• GNU APL's ? (shuffle) operator appears to have a static seed and the docs don't clearly state how to seed it: the ⎕RL system variable has the shuffle op's internal state so assigning to it will seed the PRNG, eg:

⎕RL ← +/ ⎕TS ⍝⍝ Seed ⎕RL (?) PRNG with sum of timestamp Y, M, D, H, M, S, ms

#!/usr/local/bin/apl --script --
 ⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝
⍝                                                                    ⍝
⍝ mazeGen.apl                          2022-01-07  19:47:35 (GMT-8)  ⍝
⍝                                                                    ⍝
 ⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝⍝

∇initPRNG
  ⍝⍝ Seed the internal PRNG used by APL ? operator
  ⎕RL ← +/ ⎕TS   ⍝⍝ Not great... but good enough
∇

∇offs ← cellTo dir
  ⍝⍝ Return the offset (row col) to cell which lies in compass (dir)
  offs ← ∊((¯1 0)(0 1)(1 0)(0 ¯1))[('nesw'⍳dir)]
∇

∇doMaze rc
  ⍝⍝ Main function
  0 0 mazeGen rc      ⍝⍝ Do the maze gen
  m                   ⍝⍝ output result
∇

∇b ← m isVisited coord;mr;mc
  →( ∨/ (coord[1] < 1) (coord[2] < 1) )/yes
  →( ∨/ (coord > ⌊(⍴m)÷2) )/yes
  b ← ' ' ∊ m[2×coord[1];2×coord[2]]
  →0
yes:
  b←1
∇

∇c mazeGen sz ;dirs;c;dir;cell;next
  →(c≠(0 0))/gen
init:
  c ← ?sz[1],?sz[2]
  m ← mazeInit sz
gen:
  cell ← c
  dirs ← 'nesw'[4?4]
  m[2×c[1];2×c[2]] ← ' '  ⍝ mark cell as visited
dir1:
  dir ← dirs[1]
  next ← cell + cellTo dir
  →(m isVisited next)/dir2
  m ← m openWall cell dir
  next mazeGen sz
dir2:
  dir ← dirs[2]
  next ← cell + cellTo dir
  →(m isVisited next)/dir3
  m ← m openWall cell dir
  next mazeGen sz
dir3:
  dir ← dirs[3]
  next ← cell + cellTo dir
  →(m isVisited next)/dir4
  m ← m openWall cell dir
  next mazeGen sz
dir4:
  dir ← dirs[4]
  next ← cell + cellTo dir
  →(m isVisited next)/done
  m ← m openWall cell dir
  next mazeGen sz
done:
∇

∇m ← mazeInit sz;rows;cols;r
  ⍝⍝ Init an ASCII grid of size (rows cols) which
  ⍝⍝ has all closed and unvisited cells:
  ⍝⍝
  ⍝⍝  +-+
  ⍝⍝  |.|
  ⍝⍝  +-+
  ⍝⍝
  ⍝⍝ @param sz - tuple (rows cols)
  ⍝⍝ @return m - a (rows × cols) ASCII matrix
  ⍝⍝⍝⍝

  initPRNG
  (rows cols) ← sz
  r ← ∊ (cols ⍴ ⊂"+-" ),"+"
  r ← r,∊ (cols ⍴ ⊂"|." ),"|"
  r ← (rows,(⍴r))⍴r
  r ← ((2×rows),(1+2×cols))⍴r
  r ← r⍪ (∊ (cols ⍴ ⊂"+-" ),"+")
  m ← r
∇

∇r ← m openWall cellAndDir ;ri;ci;rw;cw;row;col;dir
  (row col dir) ← ∊cellAndDir
  ri ← 2×row
  ci ← 2×col
  (rw cw) ← (ri ci) + cellTo dir
  m[rw;cw] ← ' '   ⍝ open wall in (dir)
  r ← m
∇

⎕IO←1

doMaze 9 9
)OFF

russtopia@rlm-devuan:~/GNUAPL$ workspaces/mazeGen.apl 
+-+-+-+-+-+-+-+-+-+
|     |       |   |
+-+ + +-+-+ + +-+ +
|   |       |   | |
+ +-+-+-+-+-+-+ + +
|     |     | | | |
+-+-+ +-+ + + + + +
|   |   | |   |   |
+ + +-+-+ +-+-+-+ +
| | |   |       | |
+ +-+ + + +-+ +-+ +
| |   |   |   |   |
+ + +-+-+-+ + + +-+
| |   |   | | | | |
+ +-+ + +-+ +-+ + +
|   | | |   |   | |
+ +-+ + + +-+ +-+ +
|       |         |
+-+-+-+-+-+-+-+-+-+

Parsing script arguments in GNU APL (⎕ARG)

In scripting languages one often wants to run the overall script like a standalone program, with a top 'main' function, yet also allow the script to be read into the REPL for editing, testing interactively and so on.

In Python a common way to do this is by checking the __main__ variable to determine execution scope.

GNU APL has the --script switch to enable invoking a .APL file and immediately executing it. In standard UNIX fashion the -- switch is also useful to divide GNU APL arguments (those for the interpreter itself) from those intended for the script. Using these two bits of information one can do the same thing in APL:

#!/usr/bin/env apl --script --

∇scriptFunction
    ⍝⍝ Whatever the script should do by default when run from shell
    ⎕←'This is an APL script'
∇


⍝ Parse args & run 'main' if not in REPL mode
⍝ in REPL mode, ⎕ARG[1] is the script path itself
⍝ (Recalling that by default, ⎕IO = 1)
∇checkMode;args
    args←(⎕ARG⍳⊂"--")↓⎕ARG ⍝ Drop all args up to and including "--"
    →(~(⊂'--script')∊⎕ARG)/0 ⍝ If "--script" not present, quit to allow REPL
    ⍝⍝ ⎕←"[",args[1],"]" ⍝ (just debugging to see args)
    scriptFunction ⍎∊args[2] ⍝ in --script mode, so run default func
∇

checkMode ⍝⍝ check the ⎕ARG vector for --script mode

A Hexadecimal Pronunciation Guide, by Robert A. Magnuson - Datamation Vol. 14, No. 1, Jan 1968

I recall discussing with a fellow CS student many years ago the topic of how speaking hexadecimal values seemed to be.. well, clunky. We both thought it strange no one had standardized a set of word stems like we have for the base ten numerals, 'teens', 'fortieth', and so on. All these years later I discovered, as it turns out, someone did. It just didn't catch on, for some reason.

An obscure article in Datamation, Vol. 14, No. 1 dated January, 1968 (alternate link, archive.org) contains an article by one Robert A. Magnuson, proposing an extension to English number pronunciation for hexadecimal:

Table 2. New Names for Hexadecimal Digits
A ann
B bet
C chris
D dot
E ernest
F frost
The pronunciation of the -teen's and -ty's for the new
digits is shown in Table 3. Note that the analog of the decimal pronunciation system has been used. The new name for each new digit has been chosen so that at least one of the -teen and -ty modifications is familiar sounding.

Table 3. -Teen and -Ty Pronunciation for the New Digits

1 A annteen
1 B betteen
1 C christeen
1 D dotteen
1 E ernesteen
1 F frosteen
A0 annty
B0 betty
C0 christy
D0 dotty
E0 ernesty
F0 frosty

Now 29's successor 2A can be pronounced "twenty-
ann" without the slightest tendency to confuse it with 28.
The pronunciation of C4 is "christy-four," and that of A4
is "annty-four." There is no problem in distinguishing
between 1A, "annteen" and 18, "eighteen." And 88, 8A,
A8, and AA are easily distinguished when pronounced
"eighty-eight," "eighty-ann," "annty-eight," and "annty-
ann."

Some two-digit hex numbers, each representing one byte, appear with their new pronunciations in Table 4.

Table 4. One-Byte Strings with Pronunciation
2F twenty-frost
F2 frosty-two
5B fifty-bet
3E thirty-ernest
AF annty-frost

Some four-digit hex numbers, each representing two
bytes, appear with their new pronunciations in Table 5.

Table 5. Two-Byte Strings with Pronunciation
A01C annty christeen
1ED0 ernesteen dotty
A007 annty oh-seven
DEAF dotty-ernest annty-frost
3A7D thirty·ann seventy-dot
47F0 forty-seven frosty

The problem of the values of the added digits being off
by one is now easily solved. Merely remember the "ages"
of these new-found friends. Learn that ann is 10, bet is
11, etc. without becoming confused with the fact that A
is associated with 1, B with 2, etc.

The problems of some of the hex digits being NUMERIC
and the others ALPHA on the model 29 keypunch is solved
in the following fashion. Select a particular finger of the
left hand, say, the little finger. No other finger of the left
hand is to be used. The home position of that left little
finger is on the NUMERIC key. The right hand is used for
typing 0-9 and the (numeric) comma while the left little
finger holds down the NUMERIC key. When· A - F arise,
they are typed with the left little finger-thus ensuring
that the NUMERIC key is not depressed. Return the left
finger to the home position on the NUMERIC key imme-
diately upon finishing A-F.

 

APL Keyboard Sticker Set

APL Keyboard Stickers for Laptop or Desktop

APL uses a special symbol set, which presents a barrier to entry for new users. Furthermore, there don't seem to be many options to get a keyboard with APL symbols, other than expensive specialty ones (here or here), and they're for PC desktops only; so what's a laptop user to do?

I figured I'd take matters into my own hands, and produce a run of APL keyboard stickers. These match the standard APL keyboard map used by historical APL implementations as well as GNU APL and Dyalog APL. I got the shipment at end of August, 2021, and they look pretty good on my laptop. Now available on my Tindie store. Check it out!

Black vinyl keycap stickers, suitable for any standard laptop or PC desktop keyboard.

To use these you'll need to set a custom keymap. For Linux, See my post here. For Windows, see here, here and finally here.

With some trivial setup, modern systems have no issue at all supporting APL symbols. I wonder if its successor, J, would have even come to be if the APL character set wasn't seen as an impediment prior to the 2000s. As a newcomer to APL and J, comparing the two, J just seems so much harder to parse as it uses ASCII exclusively; it looks a lot more like line noise to me whereas APL encourages seeing a unique notation.