Main

h1. Miscellaneous

h2. sshuttle

<pre>

$ sshuttle --dns -NHr dsorber@phalanx.homelinuxserver.org:3268  192.168.0.0/16

</pre>

h2. Batch Convert PNG to JPG

NOTE: @convert@ comes from ImageMagick

<pre>

& ls -1 *.png | parallel --eta convert '{}' -quality 90 '{.}.jpg'

</pre>

h2. Crop Video with ffmpeg

<pre>

$ ffmpeg -i in.mp4 -filter:v "crop=out_w:out_h:x:y" out.mp4

</pre>

* @out_w@ is the width of the output rectangle

* @out_h@ is the height of the output rectangle

* @x@ and @y@ specify the top left corner of the output rectangle (coordinates start at (0,0) in the top left corner of the input) 

Test using @ffplay@:

<pre>

$ ffplay -i input -vf "crop=out_w:out_h:x:y"

</pre>

h2. Build Custom Windows ISOs

* https://uupdump.net/

* https://forums.mydigitallife.net/threads/kms_vl_all-smart-activation-script.79535/

h2. Script to Convert Local manpages to html

Note use @zcat@ if the manpages are compressed:

<pre>

#!/bin/bash

mkdir -p html

for file in *; do

        if [ -f $file ] && [ "${file: -3}" != ".sh" ]; then

                echo $file

                #echo "${file%.*}"

                pandoc --from man --to html < $file > html/${file%.*}.html

        fi

done

</pre>

h2. Display Total rsync Progress

<pre>

$ rsync -a --no-i-r --info=progress2 /path/to/src /path/to/dest/

</pre>

NOTE: @-v@ does not work with @--info=progress2@

Source: https://serverfault.com/a/441724/472586

h2. Slice Video File 

<pre>

$ ffmpeg -ss 00:01:00 -to 00:02:00  -i input.mp4 -c copy output.mp4

</pre>

Explanation of the command:

* -i: This specifies the input file. In that case, it is (input.mp4).

* -ss: Used with -i, this seeks in the input file (input.mp4) to position.

* 00:01:00: This is the time your trimmed video will start with.

* -to: This specifies duration from start (00:01:40) to end (00:02:12).

* 00:02:00: This is the time your trimmed video will end with.

* -c copy: This is an option to trim via stream copy. (NB: Very fast)

h2. Add new drive SW RAID5

# Format drive if needed (see below for steps).

# Add new drive to SW RAID:

<pre>

$ sudo mdadm --manage /dev/md0 --add /dev/sdd1

</pre>

# Grow array:

<pre>

$ sudo mdadm --grow --raid-devices=4 /dev/md0

</pre>

h2. Replace failed SW RAID drive

# If drive is brand new be sure to create a partition on it:

<pre>

sorber@file2:~$ sudo parted /dev/sdg

GNU Parted 3.3

Using /dev/sdg

Welcome to GNU Parted! Type 'help' to view a list of commands.

(parted) mklabel gpt

(parted) mkpart primary 2048s 100%                                        

(parted) print                                                            

Model: ATA ST2000DM008-2FR1 (scsi)

Disk /dev/sdg: 2000GB

Sector size (logical/physical): 512B/4096B

Partition Table: gpt

Disk Flags: 

Number  Start   End     Size    File system  Name     Flags

 1      1049kB  2000GB  2000GB               primary

(parted) quit

</pre>

# If needed, mark bad drive as failed:

<pre>

$ sudo mdadm --manage /dev/md0 --fail /dev/sdd

</pre>

# Add new drive to replace failed:

<pre>

$ sudo mdadm --manage /dev/md0 --add /dev/sdd1 /dev/sdg1

</pre>

# Improve rebuild speed slightly:

<pre>

$ sudo bash -c "echo 9999999 > /proc/sys/dev/raid/speed_limit_min"

$ sudo bash -c "echo 9999999 > /proc/sys/dev/raid/speed_limit_max"

</pre>

# Monitor rebuild:

<pre>

$ watch -n1 cat /proc/mdstat 

</pre>

h2. Get detailed drive information

<pre>

$ udevadm info --query=all --name=/dev/sda

</pre>

h2. Split flac with cue

<pre>

$ sudo apt install cuetools shntool

$ shnsplit -f file.cue -t %n-%t -o flac file.flac

</pre>

h2. Fix Chrome Display Glitches

Somewhere around Chrome 86 display glitches started happening on wake. Here is how to fix:

Enabling @enable-vulkan@ in @chrome://flags@ then restarting the browser fixed it. If that does not work, you can also try enabling `ignore-gpu-blacklist` as well.

h2. Setup SSH Public Key

# On host machine generate a key pair. This key can be reused for multiple remote machines so it's not really necessary to change the name unless multiple keys are generated, which is also not really needed.

<pre>

$ ssh-keygen -t rsa

</pre>

 *NOTE:* for NetBeans to use/recognize the ssh key you must use the old PEM format:

<pre>

$ ssh-keygen -m PEM -t rsa

</pre>

# Now use `ssh-copy-id` to copy the key to the remote machine (for example):

<pre>

$ ssh-copy-id -i ~/.ssh/phalanx_rsa.pub dsorber@phalanx

</pre>

h2. Read Windows Product Key from ACPI Firmware

Boot into Linux and run:

<pre>

$ sudo strings /sys/firmware/acpi/tables/MSDM

</pre>

h2. Windows 10 Install and SysPrep Instructions

Very good instructions: https://theitbros.com/sysprep-windows-10-machine-guide/

h2. Search and Replace on Binary Files

A good hex editor can do search and replace on binary files but sometimes that's not practical:

* BBE - https://sourceforge.net/projects/bbe-/

 BBE is a binary editor with SED-like syntax:

<pre>

$ bbe -e 's/\x0a\x0d\x0a/\x0d\x0a/' infile > outfile

</pre>

* binwalk - https://github.com/ReFirmLabs/binwalk

 binwalk can search for hex strings (and do many other things)

<pre>

$ binwalk -R "\x0a\x0d\x0a" infile

WARNING: Signature '0    string    \x0a\x0d\x0a    Raw signature (\x0a\x0d\x0a)' is a self-overlapping signature!

DECIMAL       HEXADECIMAL     DESCRIPTION

--------------------------------------------------------------------------------

242           0xF2            Raw signature (\x0a\x0d\x0a)

412           0x19C           Raw signature (\x0a\x0d\x0a)

580           0x244           Raw signature (\x0a\x0d\x0a)

749           0x2ED           Raw signature (\x0a\x0d\x0a)

. . .

</pre>

h2. Wake-on-LAN WOL

Must be enabled on interface; need MAC

<pre>

$ sudo apt install etherwake

$ sudo etherwake <target MAC> -i <outgoing interface>

$ sudo etherwake b4:2e:99:a5:1e:f0 -i enp30s0

</pre>

h2. Allocator Shim

From Chromium, this is really interesting

* https://chromium.googlesource.com/chromium/src/base/+/master/allocator/README.md

However it is not clear how to use it outside of Chromium.

h2. Update Geany

# Install prereq packages (this obviously only needs to be done once):

<pre>

$ sudo apt install libgtk-3-dev libgtkspell-dev libgtkspell3-3-dev

</pre>

# Set compile flags:

<pre>

$ export CFLAGS="-O3 -march=native" 

$ export CXXFLAGS=$CFLAGS

</pre>

# Build geany and set to install in /opt so it won't collide with the packaged version:

<pre>

$ tar xf geany-1.30.tar.bz2

$ cd geany-1.30/

$ ./configure --prefix=/opt/local --enable-gtk3

$ make -j20

$ sudo make install

</pre>

# Set PKG_CONFIG_PATH environment variable so we can build the plugins.

<pre>

$ export PKG_CONFIG_PATH=/opt/local/lib/pkgconfig/

</pre>

# Build geany-plugins:

<pre>

$ tar xf geany-plugins-1.30.tar.bz2 

$ cd geany-plugins-1.30/

$ ./configure --prefix=/opt/local --enable-gtk3

$ make -j20

$ sudo make install

</pre>

h2. Find and replace across multiple files

<pre>

$ find /home/www -type f -print0 | xargs -0 sed -i 's/subdomainA\.example\.com/subdomainB.example.com/g'

</pre>

h2. Useful IPMI commands

Power measurement on the SuperMicro (probably won't work on Dell chassis as they have some proprietary version of ipmi):

<pre>

sudo ipmitool -U ADMIN -P ADMIN -H 10.17.136.108 dcmi power reading

</pre>

Check power status (i.e. on/off) of chassis:

<pre>

sudo ipmitool -U ADMIN -P ADMIN -H 10.17.70.45 power status

sudo ipmitool -U ADMIN -P ADMIN -H 10.17.70.45 power off

</pre>

Get IPMI IP address (look for "IP Address"):

<pre>

sudo ipmitool lan print 

</pre>

h2. Full sshfs Connect Command

<pre>

sudo sshfs -o idmap=user -o uid=$(id -u) -o gid=$(id -g) -o allow_other -o follow_symlinks ryftuser@sm-X11DGQ:/home/ryftuser /mnt/sm-X11DGQ/

</pre>

h2. Side-by-side Color Diff

<pre>

$ sudo apt-get install colordiff

$ colordiff -y --suppress-common-lines data.txt /ryftone/data.txt

</pre>

h2. x86 ABI

* https://github.com/hjl-tools/x86-psABI/wiki/X86-psABI

h2. clang-format and clang-tidy

* http://www.labri.fr/perso/fleury/posts/programming/using-clang-tidy-and-clang-format.html

h2. Shared Library and Related Resources

* http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html

* https://blog.qt.io/blog/2011/10/28/rpath-and-runpath/

* http://www.bnikolic.co.uk/blog/linux-ld-debug.html

* http://insanecoding.blogspot.com/2012/07/creating-portable-linux-binaries.html

* https://stackoverflow.com/questions/4156055/static-linking-only-some-libraries

h2. Ubuntu: List Grub Entries

<pre>

gawk  'BEGIN {                                                                                                                       

  l=0                                                                                                                                

  menuindex= 0                                                                                                                       

  stack[t=0] = 0                                                                                                                     

}                                                                                                                                    

function push(x) { stack[t++] = x }                                                                                                  

function pop() { if (t > 0) { return stack[--t] } else { return "" }  }                                                              

{                                                                                                                                    

if( $0 ~ /.*menu.*{.*/ )                                                                                                             

{                                                                                                                                    

  push( $0 )                                                                                                                         

  l++;                                                                                                                               

} else if( $0 ~ /.*{.*/ )                                                                                                            

{                                                                                                                                    

  push( $0 )                                                                                                                         

} else if( $0 ~ /.*}.*/ )                                                                                                            

{                                                                                                                                    

  X = pop()                                                                                                                          

  if( X ~ /.*menu.*{.*/ )                                                                                                            

  {                                                                                                                                  

     l--;                                                                                                                            

     match( X, /^[^'\'']*'\''([^'\'']*)'\''.*$/, arr )                                                                               

     if( l == 0 )                                                                                                                    

     {                                                                                                                               

       print menuindex ": " arr[1]                                                                                                   

       menuindex++                                                                                                                   

       submenu=0                                                                                                                     

     } else                                                                                                                          

     {                                                                                                                               

       print "  " (menuindex-1) ">" submenu " " arr[1]                                                                               

       submenu++                                                                                                                     

     }                                                                                                                               

  }                                                                                                                                  

}                                                                                                                                    

}' /boot/grub/grub.cfg

</pre>

* https://askubuntu.com/a/941993/392271

h2.GCC Disable Unused Variable Warnings

<pre>

#pragma GCC diagnostic push

#pragma GCC diagnostic ignored "-Wunused-variable"

( your problematic library includes )

#pragma GCC diagnostic pop

</pre>

See: https://stackoverflow.com/a/22708539/73878

h2. Fix Steam after NVidia driver update

Since this has now bitten me twice... Steam requires the 32 bit version of the GL library.

<pre>

$ sudo apt install libnvidia-gl-418:i386

</pre>

h2. Restart Cinnamon from TTY

# Cinnamon freezes

# Switch tty. I usually go to tty6, `Ctrl+Alt+F6`

# If you need to login first, do so.

# Type `w` (yes, just the letter) and press enter. This commands does a lot of different things, but you need it to figure out the number of the display you are using. The display number is in the column FROM. Mine is `:0` (yes, including the colon).

# Assuming that cinnamon is already dead (which you would notice by the windows lacking titles and that you can't move different windows around, and perhaps even not being able to use the keyboard), you type `export DISPLAY=:0; cinnamon &`, and don't forget the colon. I add the ampersand (&) only not to keep that tty busy.

Sauce: https://askubuntu.com/a/198935/392271

h2. Ubuntu Set Specific Kernel to Boot

* Use this command to get a list of available options:

<pre>

$ grep -Ei 'submenu|menuentry ' /boot/grub/grub.cfg | sed -re "s/(.? )'([^']+)'.*/\1 \2/"

menuentry  Ubuntu

submenu  Advanced options for Ubuntu

    menuentry  Ubuntu, with Linux 4.4.0-34-generic

    menuentry  Ubuntu, with Linux 4.4.0-34-generic (upstart)

    menuentry  Ubuntu, with Linux 4.4.0-34-generic (recovery mode)

menuentry  System setup

</pre>

* Backup grub configuration, just in case:

<pre>

$ sudo cp /etc/default/grub /etc/default/grub.bak

</pre>

* Edit grub configuration and change `GRUB_DEFAULT` to something like this:

<pre>

$ sudo vim /etc/default/grub

GRUB_DEFAULT="Advanced options for Ubuntu>Ubuntu, with Linux 3.13.0-53-generic"

</pre>

* Update grub configuration

<pre>

$ sudo update-grub

</pre>

* See also: https://askubuntu.com/questions/216398/set-older-kernel-as-default-grub-entry

h2. Ubuntu Networking Resources

* https://blog.ubuntu.com/2017/12/01/ubuntu-bionic-netplan

* https://help.ubuntu.com/lts/serverguide/network-configuration.html.en

* https://netplan.io/

h2. Win10 Licensing Info

* https://www.windowscentral.com/how-change-product-key-windows-10

* https://www.tenforums.com/tutorials/49586-determine-if-windows-license-type-oem-retail-volume.html

* https://www.tenforums.com/windows-updates-activation/104569-win-10-activation-expiring-volume-license.html

h2. Python GTK Development Resources

* https://python-gtk-3-tutorial.readthedocs.io/en/latest/layout.html

* https://lazka.github.io/pgi-docs/Gtk-3.0/classes/Box.html#Gtk.Box.pack_start

* https://lazka.github.io/pgi-docs/Gtk-3.0/mapping.html

h2. cpufrequtils Governor Configuration

These instructions are intended for Ubuntu 16.04 and later (using systemd init).

* Check the available and currently set governors. Ubuntu has an “ondemand” service that sets the governor to ondemand after boot.

<pre>

$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors

conservative ondemand userspace powersave performance schedutil

$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor 

ondemand

</pre>

* Configure cpufrequtils to set the governor to performance at boot

<pre>

$ echo 'GOVERNOR="performance"' | sudo tee /etc/default/cpufrequtils

GOVERNOR="performance"

</pre>

* Disable the automatic ondemand service. 

<pre>

$ sudo systemctl disable ondemand

</pre>

* Reboot.

* Now check that the governor is set to performance for every CPU:

<pre>

$ cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

performance

performance

performance

performance

…

performance

performance

performance

</pre>

* Check current CPU speeds:

<pre>

$ watch -n 1 bash -c 'cpufreq-info | grep "current CPU frequency"'

</pre>

h2. rsync via ssh example

<pre>

$ rsync --info=progress2 -ahz --compress-level=9 -e "ssh -i ~/ryft-pat-f1.pem" data/ ryftuser@18.205.65.5:/ryftone/regression/pubmed/data/

</pre>

h2. Read Performance using AIO

Use this fio command (run from temp directory on disk in question) to measure max disk read performance:

<pre>

$ fio --name TEST --eta-newline=5s --filename=fio-tempfile.dat --rw=read --size=10g --io_size=10g --blocksize=512k --ioengine=libaio --fsync=0 --iodepth=32 --direct=1 --numjobs=1 --runtime=300 --group_reporting

</pre>

Note that DIRECT_IO is required when using AIO. (see also: https://askubuntu.com/a/991311/392271)

h2. Remove Source Files While Creating Tarball

This is handy as it saves on disk space.

<pre>

$ tar --remove-files -cf pictures.tar Pictures/

</pre>

h2. Linux Find and Remove Files

Find and remove files (even with spaces in the path) using case insensitive matching for the wildcard:

<pre>

$ find . -type f -iname "*.jpg" -print0 | xargs -0 rm

</pre>

h2. Convert ASCII hex to binary

This problem seems to come up every so often. There is no need for a one-off solution:

<pre>

$ xxd -r -p input.txt output.bin

</pre>

https://stackoverflow.com/questions/7826526/transform-a-hex-info-to-binary-using-linux-command

h2. Well Said

https://developers.slashdot.org/comments.pl?sid=12326098&cid=56908942

h2. Hex Editor

I had been using [[https://hexinator.com/|Hexinator]] but I've had some minor issues with it. Today I tried [[https://www.wxhexeditor.org/home.php|wxHexEditor]] and I was pleasantly surprised.

Download from here: https://github.com/EUA/wxHexEditor

You'll need the latest version of libwxgtk<version>-dev package.

I had some minor issues with the build (which seems a somewhat bizarre arrangement...) modify the makefile to set `CC` and `CXX` directly, right about line 90.

Finally, to get it installed in the right place:

<pre>

sudo make install DESTDIR=/opt/local BINDIR=bin DATADIR=share

</pre>

h2. Preprocessor Fun

To invoke the preprocessor directly:

<pre>

gcc -E -DSOME_DEFINE=SOME_VALUE somefile.c 

</pre>

See also [[https://linux.die.net/man/1/unifdef|unifdef]]: "The unifdef utility acts on #if, #ifdef, #ifndef, #elif, #else, and #endif lines, and it understands only the commonly-used subset of the expression syntax for #if and #elif lines."

h2. Convert Oversampled .flac to 16 bit 44.1 khz .flac

<pre>

mkdir -p output && find . -type f -name "*.flac" | parallel -j16 "sox {} -b 16 -r 44100 output/{/.}.flac"

mkdir -p output && find . -type f -name "*.flac" | parallel -j16 "sox -v 0.98 {} -b 16 -r 44100 output/{/.}.flac"

</pre>

h2. Aligned C++ STL Types

<pre>

#include <vector>

#include <boost/align/aligned_allocator.hpp>

template <typename T>

using aligned_vector = std::vector<T, boost::alignment::aligned_allocator<T, 4096>>;

</pre>

h2. Command line convert .png to .jpg

<pre>

$ for x in *.png; do convert $x $(basename $x .png).jpg; done

</pre>

You can also add the "`-quality 90`" option if needed.

h2. Determine CPU Family Name

<pre>

$ gcc -march=native -Q --help=target|grep march

</pre>

h2. Convert .flac to .m4a (AAC)

Requires `ffmpeg` built with libfdk_aac:

<pre>

$ /opt/bin/ffmpeg -i input.flac -vn -c:a libfdk_aac -vbr 3 output.m4a

</pre>

All together using `find` and `parallel` for magic:

<pre>

$ find . -type f -name "*.flac" | parallel -j16 "/opt/bin/ffmpeg -i {} -vn -c:a libfdk_aac -vbr 4 {/.}.m4a"

</pre>

h2. PyGObject

This is the magic needed to have stuff happen quasi asynchronously using PyGObject (GTK+):

<pre>

# Yield to the main loop temporarily

while Gtk.events_pending():

    Gtk.main_iteration()

</pre>

h2. FUSE Code

 *https://pastebin.com/wavKntiu

h2. Use Windoze Schtuffs

 * https://support.microsoft.com/en-gb/help/17588/fix-problems-that-block-programs-from-being-installed-or-removed (gotta love how this requires a separate program)

 * [[http://landinghub.visualstudio.com/visual-cpp-build-tools|MSVC Tools]] - compiler (seems to be the rough equivalent of `build-essential`)

 * [[https://docs.microsoft.com/en-us/sysinternals/downloads/procmon|Process Monitor]] - useful debug tool

  * https://www.codeproject.com/Articles/560816/Troubleshooting-dependency-resolution-problems-usi - example for using Process Monitor

 * http://download.microsoft.com/download/7/2/E/72E0F986-D247-4289-B9DC-C4FB07374894/wdexpress_full.exe - (required by wingtk below)

* https://github.com/wingtk/gvsbuild - haven't tried this yet but looks amazing... 

h2. Build Handbrake Git Build

Steps for building Handbrake using AOCC:

<pre>

$ source /home/dsorber/Downloads/aocc/setenv_AOCC.sh

$ export CC=clang

$ export CXX=clang++

$ export CFLAGS="-g -O3 -mtune=native"

$ export CXXFLAGS=$CFLAGS

$ cd ~/Downloads/HandBrake/

$ rm -rf build/

$ ./configure --prefix=/opt --enable-x265 --enable-fdk-aac --gcc=/home/dsorber/Downloads/aocc/AOCC-1.1-Compiler/bin/clang

$ cd build

$ make -j17

$ sudo make install

</pre>

----

Regular build (18.04 and later):

<pre>

$ rm -rf build/

$ ./configure --prefix=/opt/local --enable-x265 --enable-fdk-aac

$ cd build

$ make -j17

$ sudo make install

</pre>

h2. Politically Correct Linux

* https://entertainment.slashdot.org/comments.pl?sid=11493671&cid=55759997

h2. This guy gets it

Regarding software purist ideology vs pragmatism

* https://linux.slashdot.org/comments.pl?sid=11156367&cid=55256437

h2. "Rotate" Video Playback via Metadata

<pre>

$ ffmpeg -i input.mp4 -c copy -metadata:s:v:0 rotate=90 output.mp4

</pre>

h2. Ubuntu Remove Old Kernels Oneliner

<pre>

$ sudo apt-get purge $(for tag in "linux-image" "linux-headers"; do dpkg-query -W -f'${Package}\n' "$tag-[0-9]*.[0-9]*.[0-9]*" | sort -V | awk 'index($0,c){exit} //' c=$(uname -r | cut -d- -f1,2); done)

</pre>

* [[https://askubuntu.com/questions/2793/how-do-i-remove-old-kernel-versions-to-clean-up-the-boot-menu/571360#571360|Sauce]]

h2. Remove New Lines (command line)

For a file:

<pre>

$ tr -d '\n' < yourfile.txt

</pre>

Or for output of a command (e.g.):

<pre>

$ find . -type f -name '*.txt' | tr -d '\n'

</pre>

h2. Convert from .mp4 to .mkv Container

Use `avconv` to convert container without re-encoding:

<pre>

$ sudo apt-get install libav-tools

$ avconv -i filme.mp4 -c copy filme.mkv

</pre>

h2. Remove String From Many File Names

On Ubuntu use Perl rename utility, e.g.:

<pre>

$ rename 's/ \(1920x1080\) \[WaifuKenny\]//' *.mp4

</pre>

h2. Change Default Text Editor

I always have to Google this...

<pre>

$ sudo update-alternatives --config editor

</pre>

h2. Linux View System RAM Speed

View RAM speed (actual) on Debian/Ubuntu/Mint:

<pre>

$ sudo lshw -short -C memory

H/W path                      Device      Class          Description

====================================================================

/0/0                                      memory         64KiB BIOS

/0/b                                      memory         32GiB System Memory

/0/b/0                                    memory         [empty]

/0/b/1                                    memory         16GiB DIMM Synchronous 2134 MHz (0.5 ns)

/0/b/2                                    memory         [empty]

/0/b/3                                    memory         16GiB DIMM Synchronous 2134 MHz (0.5 ns)

/0/d                                      memory         768KiB L1 cache

/0/e                                      memory         4MiB L2 cache

/0/f                                      memory         16MiB L3 cache

</pre>

h2. Linux Memory Schtuff

* [[https://github.com/jbert/exmap|exmap]]

* https://jameshunt.us/writings/smaps.html

* https://techtalk.intersec.com/2013/07/memory-part-2-understanding-process-memory/

* http://bmaurer.blogspot.com/2006/03/memory-usage-with-smaps.html

* https://github.com/torvalds/linux/blob/master/Documentation/filesystems/proc.txt

* From: http://stackoverflow.com/questions/24484481/entry-in-proc-meminfo

<pre>

  2. Active(file), Inactive(file) has file back-end which means its original file is in disk but to use it faster it was loaded on RAM.

</pre>

* From: http://unix.stackexchange.com/questions/56879/tracking-down-missing-memory-usage-in-linux

  The "memory used by a process" is not a clear cut concept in modern operating systems. What can be measured is the size of the address space of the process (SIZE) and resident set size (RSS, how many of the pages in the address space are currently in memory). Part of RSS is shared (most processes in memory share one copy of glibc, and so for assorted other shared libraries; several processes running the same executable share it, processes forked share read-only data and possibly a chunk of not-yet-modified read-write data with the parent). On the other hand, memory used for the process by the kernel isn't accounted for, like page tables, kernel buffers, and kernel stack. In the overall picture you have to account for the memory reserved for the graphics card, the kernel's use, and assorted "holes" reserved for DOS and other prehistoric systems (that isn't much, anyway).

  The only way of getting an overall picture is what the kernel reports as such. Adding up numbers with unknown overlaps and unknown left outs is a nice exercise in arithmetic, nothing more.

 * [[http://virtualthreads.blogspot.com/2006/02/understanding-memory-usage-on-linux.html|Understanding memory usage on Linux]]

 * https://linux-mm.org/LinuxMM

h2. Reinstall grub on UEFI System

Reinstalling grub on a UEFI system is slightly different than on a traditional legacy BIOS system:

<pre>

# Mount the primary drive root partition and the boot EFI partition

sudo mount /dev/sda# /mnt            # Mount root (/) partition

sudo mount /dev/sda# /mnt/boot       # Mount boot (/boot) partition (if separate from root partition; typically not)

sudo mkdir -p /mnt/boot/efi          # Create EFI partition mount point

sudo mount /dev/sda1 /mnt/boot/efi   # Mount EFI partition

# Mount your virtual filesystems:

for i in /dev /dev/pts /proc /sys /run; do sudo mount -B $i /mnt$i; done

# Chroot

sudo chroot /mnt

</pre>

Now, if you have networking you'll need DNS. If using `resolvconf` (for recent versions of Ubuntu):

# Edit ` /etc/resolvconf/resolv.conf.d/head` and add the following line (OpenDNS server).  Then save and exit:

<pre>

add nameserver 208.67.222.222

</pre>

# Next enable updates and update:

<pre>

sudo resolvconf --enable-updates

sudo resolvconf -u

</pre>

# Now you should have DNS. Test by pinging something.

Finally, back to fixing grub. Note that the grub install should run smoothly with no errors or warnings. If you see any something is probably wrong.

<pre>

apt-get install grub-efi-amd64  # Install grub EFI bootloader (this should be all you need to do)

# manual steps if you need them for some reason

grub-install --recheck --no-floppy --force /dev/sda1  # Install grub bootloader in EFI partition

#####

echo "configfile (hd0,gpt#)/boot/grub.cfg" > /boot/efi/ubuntu/grub.cfg # Tell grub to load grub.cfg from /boot

update-grub                          # Create grub menu list

#####

exit                                 # Exit chroot

</pre>

Reboot and you should be back in business.

h2. Convert flac to VBR mp3 using ffmpeg/lame

<pre>

for file in *.flac; do name=$(echo $file | sed "s/\.flac//g"); ffmpeg -i "$file" -codec:a libmp3lame -qscale:a 0 "$name".mp3; done

</pre>

h2. Create UEFI/legacy BIOS Bootable ISO

* http://askubuntu.com/questions/625286/how-to-create-uefi-bootable-iso

h2. Getting rEFInd to work on Mac OS 10.11

* http://mattjanik.ca/blog/2015/10/01/refind-on-el-capitan/

h2. Convert DTS 5.1 to FLAC

<pre>

sudo apt-get install libdca-utils ffmpeg sound-juicer gnome-terminal easytag vlc 

for file in *.wav; do name=$(echo $file | sed "s/\.wav//g"); dcadec -o wavall "$file" > "$name"_decoded.wav; ffmpeg -i "$name"_decoded.wav "$name".flac; rm -f "$file" "$name"_decoded.wav; done

</pre>

Source: http://ubuntuforums.org/archive/index.php/t-1849260.html

h2. Windows Utils

* http://windirstat.info/download.html

* https://www.raymond.cc/blog/safely-delete-unused-msi-and-mst-files-from-windows-installer-folder/

h2. SSH Security

https://stribika.github.io/2015/01/04/secure-secure-shell.html

h2. Windows 10

* http://ultimateoutsider.com/downloads/ - GWX Control Panel to disable GWX (Windows 10 update nagging)

* [[https://www.safer-networking.org/spybot-anti-beacon/|Spybot Anti-Beacon]] - Disable Micro$oft "telemetry" data gathering from Windows 10, 8.x, and 7

<pre>

The newest version of the KB3035583 update includes a background process which scans the 

system's Windows Registry twice a day to see if the values for the four aforementioned registry 

inputs were manually edited to disable the upgrade prompt. If they were, the process will alter 

the values, silently re-download the Windows 10 installation files (about 6 GB in total), and 

prompt the user to upgrade.

</pre>

h2. gpg-agent

* http://www.infrastructureanywhere.com/documentation/additional/mirrors.html

These instructions are geared toward reprepro but should be useful in general for setting up `gpg-agent`.

h2. Resolve DNS While VPN Connected

How to configure dnsmasq for DNS overrides while using a VPN connection: https://chawlasumit.wordpress.com/2014/06/15/linux-mint-openconnect-vpn-dns-issues/

h2. Pretty Print Python Traceback

<pre>#!python

import traceback

 ...

try: 

    with open('/dev/null', 'w') as void:

        rc = not bool(subprocess.call('modprobe -r {:s}'.format(mod_name), 

                                      shell=True, stdout=void, stderr=void))

except Exception as e:

    print_failure_status()

    exc_type, exc_value, exc_traceback = sys.exc_info()

    print('\n')

    traceback.print_exception(exc_type, exc_value, exc_traceback)

    print('\n')

</pre>

h2. Generating Random Data Files for Testing

One approach is to use /dev/urandom (non-blocking pseudorandom device):

<pre>

$ dd if=/dev/urandom bs=32M count=32 of=data_1GB.bin

</pre>

Unfortunately /dev/urandom is a bit slow. However, you can alternatively use openssl to generate pseudorandom numbers a bit faster:

<pre>

 $ openssl rand 1073741824 > data_1GB.bin

 $ for run in {1..30}; do echo $run; openssl rand 1073741824 >> data_30GB.bin; done

</pre>

h2. Cloning HDDs

When configuring multiple chassis it is necessary to clone an HDD. There are all sorts of disk imaging tools out there but it's hard to beat the trusty dd command. The pv (pipe viewer) portion isn't strictly necessary but it prints out a very useful textual status bar. The back ticked blockdev command reports the size of the drive in bytes, which pv then uses to determine overall progress. The bs argument to dd should correspond to the cache size of the drive for optimum performance (32MB is a sane default for most drives these days).

To clone a single drive to another identical drive (assuming sda is the source and sdb is the destination):

<pre>

 $ sudo dd if=/dev/sda bs=32M | pv -s `sudo blockdev --getsize64 /dev/sda` | sudo dd of=/dev/sdb bs=32M

</pre>

To clone to multiple drives all at once (assuming sda is the source and sdb, sdc are destinations):

<pre>

 $ sudo dd if=/dev/sda bs=32M | pv -s `sudo blockdev --getsize64 /dev/sda` | tee >(sudo dd of=/dev/sdb bs=32M) | sudo dd of=/dev/sdc bs=32M

</pre>

It is entirely possible to clone more drives by adding additional `>(sudo dd of=/dev/sdX bs=32M)` arguments to the tee command. Note that tee outputs to stdout so it is necessary to pipe the output of tee to the final dd command as shown above. If you don't do this then tee will spew the raw contents of the source HDD to your screen, which will make the clone take forever and look especially ugly!

For some reason RedHat/CentOS do not have the pv command or even a package for it in their repos!!! Therefore, boot from a Debian-based OS in order to clone.

h2. Generating ISOs

<pre>

 $ genisoimage -R -J -o <output.iso> <some directory>

</pre>

To deal with long file names:

<pre>

 $ genisoimage -R -J -l -D -joliet-long -o <output.iso> <some directory>

</pre>

h2. BIOS/grub

* http://forums.linuxmint.com/viewtopic.php?f=46&t=119832

* http://www.gnu.org/software/grub/manual/html_node/BIOS-installation.html

* https://wiki.archlinux.org/index.php/GRUB2

* http://unix.stackexchange.com/questions/28443/does-grub2-support-putting-boot-on-a-raid5-partition

h3. Creating a BIOS Boot Partition on a GPT system

<pre>

$ sudo parted /dev/sda

(parted) mkpart primary 0 1024k

(parted) set 2 bios_grub on

</pre>

h2. Actiontec Router

* http://transition.fcc.gov/oet/ea/fccid/

* http://opensource.actiontec.com/mi424wref.html

h2. Temp

https://www.grumpyland.com/blog/183/installing-software-raid-on-centos-567-via-ssh/

<pre>

# swap is on /dev/md1

/dev/md1 none            swap    sw              0       0

# storage is on /dev/md0

/dev/md0        /storage        xfs     noatime,barrier=0 0 0

//faramir/production/sw /sw     cifs    username=samba,password=w00tw00t!,file_mode=0666,dir_mode=0777  0       0

</pre>

----

[[../|Back home]]