==================
COMMAND LINE NOTES
==================
7-ZIP
=====
7za a -mhe -p archive-name.7z original-file
a = add to archive
-mhe = encrypt headers as well as data
-p = prompt for password
You can also add the switch
-mx0
if you do not wish to do any compressing.
For maximum compression, use:
7za a -mx9 -mhe -p archive-name.7z original-file
APT
===
apt-get --no-install-recommends install package-name
CD
==
To return to the directory that you just left:
cd -
DATE
====
FreeBSD: Set the date to 5:05 pm, January 21, 2018
date 201801211705
yyyymmddhhmm
If you only need to change the hours and minutes
date 1705
will change the time to 5:05 pm and leave the date unchanged.
TOTP codes regenerate every thirty seconds, starting at 0 and 30. To display
the current second:
date +%S
You could also run a simple script that returns the number of seconds until
the TOTP code is regenerated:
#!/bin/sh
SEC=$( date +%S )
TL1=$((30-$SEC))
TL2=$((60-$SEC))
if [ $TL2 -ge 30 ]; then
expr $TL1
else
expr $TL2
fi
If you want to get even fancier, the following script will print the seconds
in red if you have fewer than ten seconds left before the TOTP code
regenerates:
#!/bin/sh
SEC=$( date +%S )
TL1=$((30-$SEC))
TL2=$((60-$SEC))
if [ $TL2 -ge 30 ]; then
TL=$TL1
else
TL=$TL2
fi
RED="\033[1;31m"
NOCOLOR="\033[0m"
if [ $TL -ge 10 ]; then
echo $TL
else
echo "${RED}$TL${NOCOLOR}"
fi
DD
==
Overwrite with zeroes a 133 byte file:
dd if=/dev/zero of=filename count=1 bs=133
Overwrite with zeroes a 1 MB byte file:
dd if=/dev/zero of=storage-bin count=1K bs=1024
Overwrite with zeroes a 1 GB byte file:
dd status=progress if=/dev/zero of=storage-bin count=1024K bs=1024
dd status=progress if=/dev/zero of=storage-bin count=1M bs=1024
On my home system, /dev/zero can be used to generate a 10G file in about 45
seconds. By contrast, /dev/urandom will take about three minutes. Do not even
bother with /dev/random.
DIG
===
Check the mx record for yandex.com at the name server dns1.yandex.net:
dig @dns1.yandex.net yandex.com mx
Check all record types in a zone:
dig yandex.com any
Note that an increasing number of authoritative DNS servers reject requests
for type any. See bullet point 10 here:
https://www.rootusers.com/12-dig-command-examples-to-query-dns-in-linux/
On Debian-based systems dig is supplied by the package dnsutils, on FreeBSD by
bind-tools.
DMIDECODE
=========
Displays hardware information. Must be run as root. See this guide:
https://www.howtoforge.com/dmidecode-finding-out-hardware-details-without-opening-the-computer-case
To display memory information:
dmidecode -t memory
DPKG
====
List installed packages:
dpkg --get-selections
Show the status of package, PACKAGE:
dpkg-query --status PACKAGE
DU
==
du -h --max-depth=1
DUPLICITY
=========
Backup files in directory "source" to a remote server. The first time duplicity
runs it will do a full backup. Subsequently, it will do an incremental backup
of changes.
duplicity --encrypt-key gpg-key /home/user/source sftp://host//home/user/target
duplicity --encrypt-key gpg-key /home/user/source file:///home/user/local-target
duplicity restore sftp://host//home/user/backup /home/user/local-restore-directory
On incremental backups, some versions of duplicity will return the
following error message related to GnuPG:
Error processing remote manifest
This is a known and benign error message that does not indicate any failures
in the backup.
ELINKS
======
If you enable one of the color modes, then [shift]-5 will cycle through the
color schemes for that mode.
You can toggle the numbering of hyperlinks with the period "."
EMACS
=====
My Emacs notes have their own page.
FALLOCATE
=========
fallocate: Preallocate or deallocate space to a file
This command can be used to create large files faster than dd. To create an
empty 1 MB file:
fallocate -l 1M filename
The -l switch specifies the size. K=kilobytes. M=Megabytes. G=Gigabytes. The
default is bytes. More specifically, M = 1024*1024 bytes but MB = 1000*1000.
FD / FDFIND
===========
A much more user-friendly version of the traditional "find" command. Debian
has renamed the upstream binary from "fd" to "fdfind" but this change is not
mentioned in the Debian man page, which is still located at "man fd".
GIT
===
Create a local directory for git repositories. Then, in that directory,
retrieve the remote repository that you wish to work on locally:
git clone git@github.com:oldfolio/notes2e.git
Note that the above command presupposes that you have added an SSH key to your
GitHub acount. Over time, your local folder can grow quite large with the
record of changes that git keeps in the .git directory. One solution is to run
the above command in a new folder and use the new smaller folder as your
working directory.
Edit locally whatever files you wish to change. To update the remote repository:
git diff (optional, to see changes)
git add -u # This adds all files that have been updated.
git add . # This adds all files in current directory, i.e.
# untracked files will become tracked files.
git commit (or git commit -am "Update message")
git push
Add a new file:
git add FILENAME
To host a static site at Github pages, create a repository for the site. In
the root directory for the site, place a text file named CNAME. The content of
the CNAME file should simply be the domain name you wish to use for the site,
e.g. notes.oldfolio.org. Then create a CNAME record at your domain’s DNS host
that points to USERNAME.github.io:
notes 300 IN CNAME oldfolio.github.io.
You can then check the Enforce HTTPS option in your repository's settings.
To check the status of your repository:
git status
When you are away from your local folder you can still edit your site by
logging into Github and editing files there. You would just need to remember
to pull those changes into your local folder with
git pull [origin master]
GNUPG
=====
Simple symmetric file encryption:
gpg -c --cipher-algo blowfish filename.txt
Encrypt to a specific user/recipient:
gpg -e -r USER file.txt
Create a detached, ascii-enarmored signature specifying which key to use:
gpg -u key-to-use -a --output file.sig --detach-sig file.txt
Create a non-detached, ascii-enarmored signature specifying which key to use:
gpg -u key-to-use --clearsign file.txt
Verify detached signature:
gpg --verify signature.sig signed-file.txt
Export public key:
gpg -a --export {key-identifier} > public-key.asc
Export secret/private key:
gpg -a --export-secret-keys {key-identifier} > secret-key.asc
If you should ever need to edit your ~/.gnupg/gpg-agent.conf file, you will
need to reload the gpg-agent once you are finished editing.
$ gpg-connect-agent reloadagent /bye
Use extreme caution if you change the gpg-agent to pinentry-curses. Doing so
breaks the graphical version of Emacs, and I have not yet found a
work-around. If you will be working remotely with GnuPG encrypted files, you
may need to set the agent to pinentry-curses. (See the dot file above.)
Otherwise, the gpg-agent will expect a graphical environment -- and fail when
one is not present.
HTML ESCAPE SEQUENCES
=====================
& will display &
< will display <
> will display >
You might also find this useful;
Be sure not to overlook the
tag, which adds curly quotation marks as
demonstrated "here".
JOURNALCTL
==========
If the systemd journal is growing too large, you can reduce the space used
in the following way:
journalctl --rotate
journalctl --vacuum-size=100M
The rotate flag archives all the currently active journal files, and the
vacuum-size flag removes all but the most recent 100M of archived journal
files.
LN
==
ln -s target-file link-name
LOSETUP
=======
# losetup -a # List the status of all loop devices
# losetup /dev/loop0 filename # Associate loop device 0 with file filename
# losetup -d /dev/loop0 # Detach loop device
NAMEBENCH
=========
Send 128 queries to only the nameservers specified:
namebench -q 128 -O 208.67.222.222, 1.1.1.1, 8.8.8.8
NETHACK
=======
Some nethack commands:
@ = toggle autopickup
d = drop
i = open inventory
r = read (as in read a spellbook)
t = throw (as in throw a dagger)
w = wield weapon
f = fire arrows in quiver using wielded bow
Q = place arrows in quiver
S = save your game and exit
P = put on (as in put on a ring)
R = remove (as in remove a ring)
W = wear armor or shield
T = take off armor or shield
Z = cast a spell
^d = bash (as in bash a door)
#chat = talk to another character
#loot = open a container
#force = attempt to open a locked container
#untrap = rescue pet from pit
Possible ~/.nethackrc
OPTIONS=color,time,hilite_pet,menucolors,!autopickup,role=valkyrie,race=human
#OPTIONS=color,time,role=wizard,race=elf,gender=female
NETSTAT
=======
To see which TCP ports are open on your server:
netstat -ant
See, also, ss below.
OPENSSL
=======
You can use openssl for simple file encryption:
openssl enc -blowfish -a -iter 12 -in filename.txt -out filename.enc
To decrypt the output file from the above example:
openssl enc -d -blowfish -a -iter 12 -in filename.enc -out filename.txt
For decryption, notice the addition of the -d switch and the reversal of the
input and output filenames. Also, notice that all of the other options are
included. Omitting any of those options will yield a failure to decrypt.
Some ciphers that you can use here:
https://notes.oldfolio.org/dot/openssl-ciphers.txt
PASS: UNIX PASSWORD-STORE
=========================
Install under Debian:
apt-get install pass-extension-otp
The above command will install the OTP extension as well as the base
password-store utility.
Specify password-store directory in ~/.profile or ~/.mkshrc, etc.
PASSWORD_STORE_DIR=/path/to/directory
export PASSWORD_STORE_DIR
Create a new password-store database:
pass init [email-address-associated-with-GPG-key]
Enter a new account in the password-store:
pass insert -m Account-Name [or]
pass insert -m Folder/Account-Name
Edit the information for an account that already exists in the password-store:
pass edit Account-Name
Show account information:
pass show Account-Name
Add a TOTP secret key to an account:
pass otp append Account-Name
When prompted enter a key URI of the form:
otpauth://totp/acct-name?secret=SECRET-KEY
You could also just add the above URI string to the password-store entry using
the pass edit Account-Name command.
Print the current TOTP code:
pass otp code Account-Name
Remove an entry from the password-store:
pass rm Account-Name [or]
pass rm -r Folder [to delete entire folder]
RCLONE
======
Use rclone to sunchronize local files/folders with a Backblaze B2 bucket. On
your home PC you should also install the Debian backblaze-b2 utility in order
to manage your Backblaze buckets and account.
$ b2 create_bucket File-Cabinet-Master allPrivate
$ rclone config # to set up or edit the configuration of remote storage
$ rclone --progress sync /home/mm/File-Cabinet-Master b2_cabinet:File-Cabinet-Master
$ rclone --progress sync b2_cabinet:File-Cabinet-Master scw_cabinet:file-cabinet
$ rclone size b2_cabinet:File-Cabinet-Master
When synchronizing to an S2 bucket, you may want to add the --size-only flag
in order to reduce the number of requests to the remote server.
rclone sync --progress --size-only /home/mm/File-Cabinet-Master scw_cabinet:file-cabinet
In addition to commercial remote services, you can also use rclone to
synchronize over sftp to one of your own servers.
rclone --progress sync /home/mm/File-Cabinet-Master cedar_ssh:/home/mm/File-Cabinet-Master
When you set up a Backblaze B2 account as an rclone remote resource, you will
need to use an application key.
The above set of instructions allow you to synchronize using a local directory
as the source and a B2 bucket as the destination. If you wish to reverse that
and use the B2 bucket as the source and a local directory as the destination,
then use the b2 tool:
$ b2 sync --dryRun --threads 1 b2://File-Cabinet-Master/ /home/mm/File-Cabinet-Master
The default number of threads is 10. I use only one to avoid annoying others
in my household who are also using the network.
SOME RCLONE EXAMPLES
Copy a single file to a target directory:
----------------------------------------
rclone copy FILENAME remote:directory/
Notice the trailing slash following the target directory.
If you wish to rename a file when you copy it, then you would use the
*copyto* command:
rclone copyto FILENAME remote:directory/NEW-FILENAME
To mount a remote resource onto your filesystem:
-----------------------------------------------
rclone mount --vfs-cache-mode full remote: /local/mount/directory
You need the "--vfs-cache-mode full" in order to have full read-write access
to the remote resource. You also need to use the command "cmount" instead of
"mount" when you are on a Mac.
To compare source and destination:
---------------------------------
This operation makes no changes to either source or destination.
rclone check [--size-only] source/directory scw_cabinet:
To exclude a directory from a sync:
----------------------------------
rclone sync -P --exclude "/ignore-this-directory/**" /source-directory remote:target-directory
You need the double asterisk following the ignored directory in order to
ignore subdirectories as well as the primary ignored directory. A single
asterisk ignores all the files in the ignored directory but does not ignore
subdirectories.
RSYNC
=====
rsync -avuP --delete source-directory/ host:/destination-directory
Notice that the source directory HAS a trailing slash, but that the
destination directory does NOT have a trailing slash.
Hetzner storage boxes only recognize relative paths. So, your rsync command
will need to look something like:
rsync -avuP --delete local-directory/ hetzner:./directory ^ notice the dot
Synchronize a single file:
rsync -avuP source-directory/filename host:/destination-directory/
^
Notice that when synchronizing a single file a trailing slash *DOES* follow
the destination directory.
If you wish to exclude a directory
(such as, /source-directory/ignore-this-directory/)
from a sync operation, you would use a construction like the following:
rsync -avuP --delete --exclude 'ignore-this-directory' /source-directory/ remote:/target-directory
SECURE_DELETE (FreeBSD) / SECURE-DELETE (Debian)
================================================
Overwrite with random data and delete all files and subdirectories of DIRECTORY
srm -llr DIRECTORY
Overwrite with zeroes and delete all files and subdirectories of DIRECTORY
srm -llzr DIRECTORY
Under OpenBSD, the standard
rm
command followed the -P switch overwrites files once with random data
before deleting. Add the -R switch to remove the entire file hierarchy,
including subdirectories.
SMEM
====
Report chrome or chromium total memory usage:
smem -t -k -c pss -P chrom | tail -n 1
Report dropbox total memory usage:
smem -t -k -c pss -P dropb | tail -n 1
Report firefox total memory usage:
smem -t -k -c pss -P firef | tail -n 1
Report opera total memory usage:
smem -t -k -c pss -P opera | tail -n 1
Report yandex browser total memory usage:
smem -t -k -c pss -P yandex_b | tail -n 1
Report yandex disk total memory usage:
smem -t -k -c pss -P yandex-d | tail -n 1
Report vivaldi total memory usage:
smem -t -k -c pss -P vivaldi | tail -n 1
SS
==
The ss command is a successor to netstat. (See netstat above.) As long as
netstat is available it is still a useful tool.
ss -at
SSH
===
Creating an SSH tunnel:
ssh -D 5222 remote-server -N
-D = bind port
-N = do not execute a remote command
chromium --proxy-server=socks5://localhost:5222
To use with Firefox, Pale Moon, etc.:
Preferences -> Advanced -> Network -> Connection -> Settings
Manual proxy configuration
SOCKS Host: 127.0.0.1 Port: 5222
No Proxy for:
localhost, 127.0.0.1
SSHFS
=====
If you install sshfs, you can mount your remote servers as an ordinary
user. Use the mount options uid and gid so that the remote directory will
belong to the local user.
$ sshfs server-nickname:/home/username /local/mountpoint -o uid=1000,gid=1000
To unmount:
$ fusermount -u /local/mountpoint
SYSCTL
======
Report hardware information on FreeBSD systems:
# sysctl hw.model hw.machine hw.ncpu
TAR
===
To archive your /etc and /home directories:
# tar cvf /root/etc-home.tar /etc /home
To create an archive that excludes some files in the target:
tar cvf ~/archive.tar --exclude='excluded-directory/*' *
To list the files in an archive:
tar tvf archive.tar
The output of listing an archive's contents will look something
like:
-rw-r--r-- user/group 1567 2019-12-12 10:50 ./file1.txt
-rw-r--r-- user/group 1997 2019-12-12 10:50 ./file2.txt
You can remove an unwanted file from an archive in the following
way:
tar --delete -f archive.tar ./file2.txt
where file2.txt is the unwanted file. You should note though that
the --delete switch will not work on compressed archives.
Compression:
bzip2 = j
gzip = z
xz = J
Create an archive with a time stamp in the archive name:
suffix=`date +%F-%H.%M`
tar cvf /home/user/archive-$suffix.tar /path/to/target-directory/
TMUX
====
Ctrl-b to enter commands
Detach the current session:
Ctrl-b d
Re-attach a previous session:
tmux attach -t 0
where "0" is the name of the previous session.
A tmux beginner's guide.
A tmux cheat sheet.
USERMOD
=======
To change a user's primary login group:
usermod -g primarygroupname username
To add a user to a secondary group:
usermod -a -G secondarygroupname username
Using the -G switch without the -a switch will remove a user from all
secondary groups except those specified by the current instance of the -G
switch.
Change a user's username:
usermod --login new-user-name --move-home --home /home/new-home-directory old-user-name
When you change a user's username you will likely also want
to change the name of the user's primary group:
groupmod --new-name new-group-name old-group-name
In most cases, the new-group-name will be the same as the new-user-name, and
the old-group-name will be the same as the old-user-name.
VIM
===
Find each occurrence of 'foo' and replace it with 'bar':
:%s/foo/bar/g
When you need vim to behave like traditional vi:
vim -u NONE -C
The -u switch specifies which vimrc file to use, with the NONE argument
instructing vim not to load any vimrc initializations. The -C switch instructs
vim to behave in a way that is compatible with traditional vi. The -C switch
by itself does not work because without "-u NONE" vim will respect your vimrc
initializations.
Edit a remote file:
vim scp://user@server.com:22//home/user/filename
or
:e scp://user@server.com:22//home/user/filename # OR
:e scp://SSH-Config-Host//home/user/filename # OR
:e scp://SSH-Config-Host/filename # /home/user not needed because you
# are automatically logged into that
# directory
Browse a remote directory:
:e scp://user@server.com:22//home/user/
:e scp://SSH-Config-Host// # Directory specification not needed if
# you wish to browse the directory you are
# initially logged into.
Prompt for an encryption key:
:X
Center text [based on a 75 character-wide line]:
:ce [75]
Set the maximum number of characters on a line to 75
set tw=75
Various editing tasks:
dd delete current line
~ switch case of characters (from CAPITALS to lower case or vice VERSA)
U MAKE ALL SELECTED CHARACTERS CAPITALS/UPPER CASE
u make all selected characters lower case
J join next line to the current one
> indent selected lines
gq apply text formatting to selected region
" specify a register
"+ specify the clipboard
"+y copy to clipboard
"+d cut to clipboard
"+P paste from clipboard before cursor
"+p paste from clipbaord after cursor
=============
Miscellaneous
=============
echo 3 > /proc/sys/vm/drop_caches
Do a searchon "drop_caches" for additional information, including the
differences between echo 1, echo 2, and echo 3.
File Size Comparison
====================
I keep all of my notes in a very large text file. Here is a size comparison of
the notes in different formats (October 2018):
RAW TEXT 2708583 bytes (100.00%)
DOCX 1254433 bytes ( 46.31%)
ODT 1103976 bytes ( 40.76%)
GZ TEXT 1040337 bytes ( 38.41%)
XZ TEXT 796340 bytes ( 29.40%)
BZ2 TEXT 763485 bytes ( 28.19%)
Browser Memory Usage
====================
PSS as reported by smem under my test scenario Jan 19, 2021:
Chromium: 360M
Firefox: 520M
Palemoon: 180M
Vivaldi: 210M
And, a second test on Jan 20, 2021:
Chromium: 215M
Firefox: 523M
Palemoon: 258M
Vivaldi: 210M
Shell Notes
===========
Memory usage reported by htop for different shells:
FREEBSD 11.3
bash 7840 3956
csh 7412 3800
ksh93 8232 4196
mksh 6608 2692
sh 7068 3064
tcsh 7412 3804
DEBIAN 10.1
bash 7599 4236
dash 2388 700
lksh 616 356
mksh 3164 2144 (non-static)
mksh 848 580 (static)
tcsh 6656 3288
Ping Speed
==========
Average ping speed from home to servers December 2020.
almond: 26.698
birch: 155.023
cedar: 45.158
elm: 47.287
fir: 27.191
larch: 45.240
teak: 65.452
SD: 48.601
Average ping speed from home to servers July 2020.
birch: 153.272
cedar: 71.759
fir: 34.283
larch: 92.014
pine: 145.663
SD: 44.081