config.toml -> pkg.toml, get cleanup working

This commit is contained in:
Reid D. 'arrdem' McKenzie 2022-02-15 10:22:53 -07:00
parent 00f2ba3bd7
commit cd9530ad66
12 changed files with 116 additions and 29 deletions

View file

@ -4,61 +4,116 @@
An alternative to GNU Stow, more some notion of packages with dependencies and install scripts.
Think an Ansible or Puppet but anyarch and lite enough to check in with your dotfiles.
Think an Ansible, Puppet or even NixOS but anyarch and lite enough to check in with your dotfiles.
## Overview
Somewhat like Stow, Cram operates in terms of packages, which are directories with the following structure -
Cram operates on a directory of packages called `packages.d/`, and two directories of metapackages called `profiles.d` and `hosts.d`.
```
/REQUIRES # A list of other packages this one requires.
/BUILD # 1. Perform any compile or package management tasks.
/PRE_INSTALL # 2. Any other tasks required before installation occurs.
/INSTALL # 3. Do whatever constitutes installation.
# This supercedes the default copying of files.
/POST_INSTALL # 4. Any cleanup or other tasks following installation
... # Any other files are treated as package contents.
### Packages
A Cram package consists of a directory containing a `pkg.toml` file with the following format -
```toml
[cram]
version = 1
[package]
# The package.require list names depended artifacts.
[[package.require]]
name = packages.d/some-other-package
# (optional) The package.build list enumerates either
# inline scripts or script files. These are run as a
# package is 'built' before it is installed.
[[package.build]]
run = some-build-command
# (optional) Hook script(s) which occur before installation.
[[package.pre_install]]
run = some-hook
# (optional) Override installation scrpt(s).
# By default, everthing under the package directory
# (the `pkg.toml` excepted) treated is as a file to be
# installed and stow is emulated using symlinks.
[[package.install]]
run = some-install-command
# (optional) Hook script(s) which after installation.
[[package.post_install]]
run = some-other-hook
```
Cram reads a config dir with three groups of packages
- `packages.d/<packagename>` contains a package that installs but probably shouldn't configure a given tool, package or group of files.
Configuration should be left to profiles.
- `profiles.d/<profilename>` contains a profile; a group of related profiles and packages that should be installed together.
- `hosts.d/<hostname>` contains one package for each host, and should pull in a list of profiles.
- Both profiles and hosts entries may specify their own "inline" packages as a convenience.
To take a somewhat real example from my own dotfiles -
The intent of this tool is to keep GNU Stow's intuitive model of deploying configs via symlinks, and augment it with a useful pattern for talking about "layers" / "packages" of related configs.
```shell
$ tree -a packages.d/tmux
packages.d/tmux
├── pkg.toml
└── .tmux.conf
```
Cram installs the package `hosts.d/$(hostname)`, and `profiles.d/default` by default.
This TMUX package provides only my `.tmux.conf` file, and a stub `pkg.toml` that does nothing.
A fancier setup could use `pkg.toml` to install TMUX either as a `pre_install` task or by using a separate TMUX package and providing the config in a profile.
### Metapackages
Writing lots of packages gets cumbersome quickly, as does managing long lists of explicit dependencies.
To try and manage this, Cram provides metapackages - packages which contain no stowable files, but instad contain subpackages.
To take a somewhat real example from my own dotfiles -
```shell
$ tree -a -L 1 profiles.d/macos
profiles.d/macos
├── pkg.toml
├── emacs/
├── homebrew/
└── zsh/
```
The `profiles.d/macos` package depends AUTOMATICALLY on the contents of the `profiles.d/macos/emacs`, `profiles.d/macos/homebrew` and `profiles.d/macos/zsh` packages, which are normal packages.
These sub-packages can have normal dependencies on other packages both within and without the profile and install files or run scripts.
Profiles allow users to write groups of related packages, especially configs, which go together and allows for scoped reuse of meaningful names.
Likewise the `hosts.d/` tree allows users to store host-specific packages.
## Usage
```
$ cram apply [--dry-run|--execute] [--optimize] <configdir> <destdir>
$ cram apply [--dry-run|--execute] [--optimize] [--require <package>] <configdir> <destdir>
```
The `apply` task applies a configuration to a destination directory.
The most common uses of this would be `--dry-run`, which functions as a `diff` or `--execute ~/conf ~/` for emulating Stow and installing dotfiles.
The most common uses of this would be `--dry-run` (the default), which functions as a `diff` or `--execute ~/conf ~/` for emulating Stow and installing dotfiles.
By default `cram` installs two packages - `profiles.d/default` and `hosts.d/$(hostname -s)`.
This default can be overriden by providing `--require <package>` one or more times to enumerate specific packages to install.
Cram always reads the `.cram.log` state file and diffs the current state against the configured state.
Files and directories no longer defined by the configured state are cleaned up automatically.
```
$ cram show <configdir>
$ cram state <configdir>
```
The `list` task loads up and prints the `.cram.log` state file generated by any previous `cram apply --execute` so you can read a manifest of what cram thinks it did.
The `state` task loads up and prints the `.cram.log` state file generated by any previous `cram apply --execute` so you can read a manifest of what cram thinks it did.
This is useful because `cram` attempts to optimize repeated executions and implement change detection using the state file.
This cache can be busted if needed by using `apply --execute --no-optimize`, which will cause cram to take all actions it deems presently required.
This can result in dangling symlinks in the filesystem.
```
$ cram list <configdir>
$ cram list <configdir> [package]
```
The `show` task lists out all available packages (eg. packages, profiles, hosts, and subpackages) as a dependency graph.
The `list` task lists out all available packages (eg. packages, profiles, hosts, and subpackages) as a dependency graph.
When provided a specific package, the details of that package (its requirements and installation task log) will be printed.
## License
Copyright Reid 'arrdem' McKenzie, 31/10/2021.
Copyright Reid 'arrdem' McKenzie, 15/02/2022.
Published under the terms of the MIT license.
See the included `LICENSE` file for more.
Published under the terms of the Anticapitalist Software License (https://anticapitalist.software).

View file

@ -0,0 +1,4 @@
__version__ = "0.1.0"
__author__ = "Reid D. 'arrdem' McKenzie <me@arrdem.com>"
__copyright__ = "Copyright 2020"
__license__ = "https://anticapitalist.software/"

View file

@ -8,6 +8,7 @@ import pickle
import sys
from typing import List
from . import __version__, __author__, __license__, __copyright__
from .v0 import PackageV0, ProfileV0
from .v1 import PackageV1, ProfileV1
@ -148,6 +149,21 @@ def scrub(old_fs: Vfs, new_fs: Vfs) -> Vfs:
@click.group()
@click.version_option(version=1, message=f"""Cram {__version__}
Documentation
https://github.com/arrdem/source/tree/trunk/projects/cram/
Features
- 0.0.0 legacy config format
- 0.1.0 TOML config format
- 0.1.0 log based optimizer
- 0.1.0 idempotent default for scripts
About
{__copyright__}, {__author__}.
Published under the terms of the {__license__} license.
""")
def cli():
pass
@ -175,7 +191,7 @@ def do_apply(confdir, destdir, state_file, execute, optimize, require, exec_idem
# Middleware processing of the resulting filesystem(s)
executable_fs = scrub(old_fs, new_fs)
if optimize:
executable_fs = simplify(old_fs, new_fs,
executable_fs = simplify(old_fs, executable_fs,
exec_idempotent=exec_idempotent)
# Dump the new state.

View file

@ -10,6 +10,7 @@ from vfs import Vfs
SHELL = "/bin/sh"
def sh(cmd: List[str], /,
env: Optional[dict] = None):

View file

@ -26,7 +26,7 @@ def tempf(name):
class PackageV1(Package):
"""The v1 package format."""
SPECIAL_FILES = ["config.toml"]
SPECIAL_FILES = ["pkg.toml"]
_config = None
def config(self):

View file

@ -83,3 +83,14 @@ rm -r "${dest}"/*
# If multiple packages provide the same _effective_ script, do it once
./cram apply --require packages.d/p6 --require packages.d/p7 --execute test/ "${dest}" | sort | uniq -c | grep "/tmp/stow/b5bea41b6c623f7c09f1bf24dcae58ebab3c0cdd90ad966bc43a45b44867e12b.sh" | grep "1 - exec"
# Test log-based cleanup
./cram apply --require packages.d/p1 --require packages.d/p2 --execute test/ "${dest}"
[ -L "${dest}"/foo ]
[ -L "${dest}"/bar ]
# And how bar shouldn't be installed...
./cram state test/
./cram apply --require packages.d/p1 --execute test/ "${dest}"
./cram state test/
[ -L "${dest}"/foo ]
[ ! -L "${dest}"/bar ]