From the 2017-09-19 BALUG.org meeting (references, etc. on some of what was discussed)
Some references, on some of the bits mentioned ...
BALUG wiki:
https://www.wiki.balug.org/wiki
It does contain more than just BALUG stuff.
Some of the stuff is rather to quite out-of-date, but also, much of
it too, is rather to quite current and maintained/updated pretty well.
As location that's automagically indexed, I typically start here:
https://www.wiki.balug.org/wiki/doku.php?do=index&idx=balug
DreamHost.com hosting exodus & email/list migrations, etc.:
https://www.wiki.balug.org/wiki/doku.php?id=balug:dreamhost_exodus
https://www.wiki.balug.org/wiki/doku.php?id=balug:mail_and_lists
Also maintain list of (mostly Linux) ISO images I have:
https://www.wiki.balug.org/wiki/doku.php?id=balug:cds_and_images_etc
bup: It backs things up
I've not tried it, but at least what I saw of it several years ago,
looked quite promising, and, at quick search/glance, looks to be
reasonably maintained/active/etc.:
https://gitlab.com/SjoenH/bup
https://groups.google.com/forum/#!forum/bup-list
loopback devices
losetup(8) (files are /dev/loop*)
"loopback" can be confusing, as it's used in quite different ways for
very different things in contexts of Linux/Unix.
There's networking loopback device/address.
SunOS/Solaris also has loopback filesystem mounts ... but that's
something quite different - it functions quite like Linux's bind mounts.
All not to be confused with Linux loopback devices.
At quick peek at man page, looks like encryption bit has been
pushed out of loopback devices and losetup, and into dmsetup(8)
One can also automagically (de)allocate loop devices with mounts,
e.g. one wants to mount a (regular) file, but mount(8) requires a block
device. Well, ...
# mount [-t type] -o loop[,...] file mount_point
The loop option to mount will create loop device and mount that, and
will remove the loop device when it's umount(8)ed. It will also show
the mount in slightly more human-friendly form if one uses the mount(8)
command to display such - as opposed to manually creating the loop
device and directly mounting the loop device itself - or likewise cat(1)
of /proc/mounts. (mount(8)/mount(2) tracks that additional information
in /etc/mtab - which isn't exactly intended to be human readable, but
intended to be read/written by mount(8)/mount(2)).
Some of the other bits I wasn't thinking of names of
off-the-top-of-my-head (often more rolls off the fingertips, than
tongue):
dmsetup(8) - some more interesting and complex device mapping,
notably file(s)/device(s) - or portions thereof, possibly including
encryption, RAID, etc. It's lower-level facility used by, e.g.:
cryptsetup(8), lvm(8), partx(8), mdadm(8), etc. Most of the time
one doesn't use dmsetup(8) - at least directly ... but once in a
while one may (e.g. to "repair" something, e.g. a drive with a
fair bunch of at least moderately complex setup (e.g. LVM, or
LUKS encryption, or both) is connected, mounted, in use ...
drive gets powered off or disconnected and then reconnected and
powered on again ... cleaning that state up to a point where one
can continue and have such mounted again in such a scenario
may effectively require some use of dmsetup(8) - and possibly also
some "lazy" unmounts (umount -l)), and maybe a bit of rm(1),
and stuff like:
# echo 1 > /sys/block/sde/device/delete
and maybe some bits of rescanning, e.g.:
# (>>/dev/null 2>&1 ls -d /sys/class/scsi_host/host*/scan && for tmp in /sys/class/scsi_host/host*/scan; do echo '- - -' > "$tmp"; done)
etc.
partx(8)
Very handy for creating(/removing) partition devices for a partitioned
(block) device, but especially where it's such a device where the kernel
and udev, etc. wouldn't typically automagically create parition devices.
E.g. /dev/sd[a-z]... would generally automagically have device(s) for
partitions, but, if, e.g. one has /dev/loop3 as a block device that
references a file that's a paritioned disk image, and one wants devices
to access those partitions ...
Here's example, setting up a paritioned disk image within a file:
# mkdir /var/tmp/partx && cd /var/tmp/partx
# truncate -s `expr 50 \* 1024 \* 1024` img
# losetup --show -f `pwd -P`/img
/dev/loop0
# fdisk /dev/loop0
Welcome to fdisk (util-linux 2.25.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0xab8c7754.
Command (m for help): n
Partition type
p primary (0 primary, 0 extended, 4 free)
e extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-102399, default 2048):
Last sector, +sectors or +size{K,M,G,T,P} (2048-102399, default 102399): 51198
Created a new partition 1 of type 'Linux' and of size 24 MiB.
Command (m for help): n
Partition type
p primary (1 primary, 0 extended, 3 free)
e extended (container for logical partitions)
Select (default p):
Using default response p.
Partition number (2-4, default 2):
First sector (51199-102399, default 51200):
Last sector, +sectors or +size{K,M,G,T,P} (51200-102399, default 102399):
Created a new partition 2 of type 'Linux' and of size 25 MiB.
Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Re-reading the partition table failed.: Invalid argument
The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8).
# ls /dev/loop0*
/dev/loop0
# partx -a /dev/loop0 && ls /dev/loop0?*
/dev/loop0p1 /dev/loop0p2
# mkfs -t ext3 /dev/loop0p1
mke2fs 1.42.12 (29-Aug-2014)
Discarding device blocks: done
Creating filesystem with 24572 1k blocks and 6144 inodes
Filesystem UUID: fbf8ed36-25d8-4d91-a62d-080b8b4199f0
Superblock backups stored on blocks:
8193
Allocating group tables: done
Writing inode tables: done
Creating journal (1024 blocks): done
Writing superblocks and filesystem accounting information: done
# mkfs -t ext3 /dev/loop0p2
mke2fs 1.42.12 (29-Aug-2014)
Discarding device blocks: done
Creating filesystem with 25600 1k blocks and 6400 inodes
Filesystem UUID: 9485beff-4240-4214-b919-5ea8c3602b64
Superblock backups stored on blocks:
8193, 24577
Allocating group tables: done
Writing inode tables: done
Creating journal (1024 blocks): done
Writing superblocks and filesystem accounting information: done
# mkdir loop0p1 loop0p2
# mount /dev/loop0p1 loop0p1 && mount /dev/loop0p2 loop0p2
# df -h loop0p*
Filesystem Size Used Avail Use% Mounted on
/dev/loop0p1 23M 209K 21M 1% /var/tmp/partx/loop0p1
/dev/loop0p2 24M 316K 22M 2% /var/tmp/partx/loop0p2
# mount | fgrep loop0p
/dev/loop0p1 on /var/tmp/partx/loop0p1 type ext3 (rw,relatime,data=ordered)
/dev/loop0p2 on /var/tmp/partx/loop0p2 type ext3 (rw,relatime,data=ordered)
# ls -ons img
4116 -rw------- 1 0 52428800 Sep 20 08:07 img
# umount loop0p2 && umount loop0p1 && rmdir loop0p2 loop0p1 &&
> partx -d /dev/loop0 && losetup -d /dev/loop0 && rm img &&
> cd && rmdir /var/tmp/partx
#
Notice also that the file was sparse - 50MiB of logical space, only
about 4.1MiB of physical blocks used.
Anyway, stuff like that can be handy for mucking about with a
"disk image" that resides in a (file of type ordinary) file.
So, for example, one can prepare a disk image within a file,
and then blast (dd(1)) it to a USB or SD flash device, or copy
image from disk to file, then work with the file copy, etc.