Project

General

Profile

Actions
History

Wiki »

Miscellaneous

Nextcloud fix "Another process is already scanning... " error

  1. Enter maintenance mode:
    $ cd /var/www/nextcloud
$ sudo -u www-data php ./occ maintenance:mode --on
  2. Connect to Nextcloud database using psql:
    $ sudo -u postgres psql -h localhost -U nextcloud nextcloud
  3. Delete the contents of the oc_file_locks table:
    > DELETE FROM oc_file_locks WHERE true;
> \q
  4. Exit maintenance mode then reattempt the files scan:
    $ cd /var/www/nextcloud
$ sudo -u www-data php ./occ maintenance:mode --off
$ sudo -u www-data php ./occ files:scan --all

Encode Animated gif to Video

Using ffmpeg:

$ /opt/local/bin/ffmpeg -hide_banner -r 24 -i rapidsave.com_q9qfgjk7w7od1.gif -movflags faststart -c:v libx264 -preset slow -b:v 1M -pix_fmt yuv420p -vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" output.mp4

# Find all gifs > 2MB and convert in parallel
$ find . -name '*.gif' -size +2M | parallel --eta -q /opt/local/bin/ffmpeg -hide_banner -r 24 -i '{}' -movflags faststart -c:v libx264 -preset slow -b:v 1M -pix_fmt yuv420p -vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" '{.}.mp4'

Recover SW RAID

$  sudo mdadm --stop /dev/md0
$  sudo mdadm --assemble --force /dev/md0 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 /dev/sdn1 /dev/sdo1 /dev/sdp1 /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1 /dev/sdu1 /dev/sdv1 /dev/sdw1 /dev/sdx1 /dev/sdy1 /dev/sdz1 /dev/sdaa1 /dev/sdac1 /dev/sdad1 /dev/sdae1 /dev/sdaf1 /dev/sdag1 /dev/sdah1 /dev/sdai1 /dev/sdaj1 /dev/sdak1 /dev/sdal1 /dev/sdam1 /dev/sdan1 /dev/sdao1 /dev/sdap1 /dev/sdaq1 /dev/sdar1 /dev/sdas1 /dev/sdat1 /dev/sdau1 /dev/sdav1 /dev/sdaw1
$ sudo mdadm -D /dev/md0
$ lsblk 
$ sudo mdadm --examine /dev/sdar1 
$ sudo mdadm --examine /dev/sdat1 
$ sudo mdadm --add /dev/md0 /dev/sdar1 /dev/sdat1
$ cat /proc/mdstat
$ sudo bash -c "echo 9999999 > /proc/sys/dev/raid/speed_limit_min" 
$ sudo bash -c "echo 9999999 > /proc/sys/dev/raid/speed_limit_max"

Fix some WIndows 11 Annoyances

  1. Install without Micro$oft Account:
    • Follow the Windows 11 install process until you get to the "choose a country" screen.
    • Hit Shift + F10. A command prompt appears.
    • Type: OOBE\BYPASSNRO to disable the Internet connection requirement. The computer will reboot afterwards.
    • Hit Shift + F10 again and this time Type: ipconfig /release. Then hit Enter to disable the Internet.
    • Continue with the installation, choosing the region. keyboard and second keyboard option. A screen saying "Let's connect you to a network" appears, warning you that you need Internet. Click "I don't have Internet" to continue. Click Continue with limited setup if prompted. The OOBE may skip this screen and take you straight to the next step.
  2. Restore old right click menu
    • Open Terminal
    • Type this: reg.exe add "HKCU\Software\Classes\CLSID\{86ca1aa0-34aa-4e8b-a509-50c905bae2a2}\InprocServer32" /f /ve
    • Restart machine.
    • To undo: reg.exe delete "HKCU\Software\Classes\CLSID\{86ca1aa0-34aa-4e8b-a509-50c905bae2a2}" /f
  • NOTE: Apparently BYPASSNRO is being removed by Micro$oft. For now this should work:
    • Hit Shift + F10
    • Type: start ms-cxh:localonly

Update Nextcloud External Files

$ cd /var/www/nextcloud
$ sudo -u www-data php ./occ files:scan --all

Change Windows 10 Version

  • Disconnect from internet
  • Go to "Settings" > "Update & Security" > "Activation" > "Change productkey"
  • Enter VK7JG-NPHTM-C97JM-9MPGT-3V66T
  • Press "Ok" and wait for the process to run
  • Reboot after error message is displayed

Codes for all editions:

[cmi]
namespace=urn:schemas-microsoft-com:unattend
ClientServerSplit=true
AnswerFileFormat=1
professionalworkstationn=WYPNQ-8C467-V2W6J-TX4WX-WT2RQ
enterpriseneval=MNXKQ-WY2CT-JWBJ2-T68TQ-YBH2V
serverazurecor=QN7G3-4RM92-MT6QR-PR966-FVYV7
serverrdshCore=NJCF7-PW8QT-3324D-688JX-2YV66
onecoreupdateos=NFDPX-3MV8X-THC2X-QQ9QP-P9YY6
serverstandardeval=BGHNQ-64HFC-WD7WG-K2T34-8477T
serverdatacenterevalcorCore=RNJ27-KR4TJ-DX6FB-XFJJ2-2DQMJ
iotenterprisek=P8Q7T-WNK7X-PMFXY-VXHBG-RRK69
ServerStandardEvalCor=KGKTQ-NWWW9-Q8KKW-4M2R9-66TFB
iotenterprise=XQQYW-NFFMW-XJPBH-K8732-CKFFD
educationn=84NGF-MHBT6-FXBX8-QWJK7-DRR8H
serverdatacentereval=K4NJQ-WBJ89-PJGPP-9MTY4-R3QK2
serverazurecorCore=QN7G3-4RM92-MT6QR-PR966-FVYV7
enterpriseseval=Y22YQ-HNV92-HWMXQ-PMDQ9-7XRGJ
coresinglelanguage=BT79Q-G7N6G-PGBYW-4YWX6-6F4BT
professionalcountryspecific=3XP6D-CRND4-DRYM2-GM84D-4GG8Y
core=YTMG3-N6DKC-DKB77-7M9GH-8HVX7
coresystemserver=6DWFN-9DBPB-99W4C-XYWKQ-VXPFM
serverdatacenterevalcor=RNJ27-KR4TJ-DX6FB-XFJJ2-2DQMJ
serverstandardcor=B2NYV-R6JT2-H9YTQ-VQ4X3-979YK
serverdatacentercorCore=DCXHG-67NR6-6CRMW-2PYFW-PKGD7
serverazurenano=9JQNQ-V8HQ6-PKB8H-GGHRY-R62H6
coren=4CPRK-NM3K3-X6XXQ-RXX86-WXCHW
EnterpriseSNEval=VW76H-NXCYD-PPWKF-JG63K-KQFGM
starter=D6RD9-D4N8T-RT9QX-YW6YT-FCWWJ
professional=VK7JG-NPHTM-C97JM-9MPGT-3V66T
professionaln=2B87N-8KFHP-DKV6R-Y2C8J-PKCKT
enterprise=XGVPP-NMH47-7TTHJ-W3FW7-8HV2C
education=YNMGQ-8RYV3-4PGQ3-C8XTP-7CFBY
serverwebCore=3WN48-GVPXF-VHWK4-29Y8D-KKTD2
ServerTurbine=3Y8N7-3R679-4XFFH-6BTCM-X4W7X
serverazurestackhcicor=YP2N4-KHPFV-F2D34-PXFJH-MBK3B
enterprisegn=FW7NV-4T673-HF4VX-9X4MM-B4H4T
enterpriseg=FV469-WGNG4-YQP66-2B2HY-KD8YX
serverweb=3WN48-GVPXF-VHWK4-29Y8D-KKTD2
serverrdsh=NJCF7-PW8QT-3324D-688JX-2YV66
serverdatacenternanoCore=BFM6D-TKNBY-X6RGR-PHJ4G-723B6
ServerDatacenter=9DYFC-FNGM2-KQGHQ-BTRK4-JHYT3
serverstandard=J768X-NXC7R-GFB9Q-BHF4H-GCVBB
enterprisesn=2D7NQ-3MDXF-9WTDT-X9CCP-CKD8V
serverdatacenternano=BFM6D-TKNBY-X6RGR-PHJ4G-723B6
ServerStandardEvalCorCore=KGKTQ-NWWW9-Q8KKW-4M2R9-66TFB
serverstandardnano=69NHX-WQQ7G-QVBPG-4HPQP-8XDB7
cloudeditionn=K9VKN-3BGWV-Y624W-MCRMQ-BHDCD
professionaleducation=8PTT6-RNW4C-6V7J2-C2D3X-MHBPB
serverazurestackhcicorCore=YP2N4-KHPFV-F2D34-PXFJH-MBK3B
serverstandardevalCore=BGHNQ-64HFC-WD7WG-K2T34-8477T
serverturbinecorCore=H7XNC-JYM86-7B27X-8MJ9W-TKFX9
professionaleducationn=GJTYN-HDMQY-FRR76-HVGC7-QPF8P
professionalworkstation=DXG7C-N36C4-C4HTG-X4T3X-2YV77
enterprises=PJB47-8PN2T-MCGDY-JTY3D-CBCPV
cloudedition=KY7PN-VR6RX-83W6Y-6DDYQ-T6R4W
iotenterprisesk=N979K-XWD77-YW3GB-HBGH6-D32MH
startern=3NFXW-2T27M-2BDW6-4GHRV-68XRX
EnterpriseEval=VPMWD-PVNRR-79WJ9-VVJQC-3YH2G
serverturbinecor=H7XNC-JYM86-7B27X-8MJ9W-TKFX9
iotenterprises=CGK42-GYN6Y-VD22B-BX98W-J8JXD
ServerDatacenterCore=9DYFC-FNGM2-KQGHQ-BTRK4-JHYT3
wnc=TMP2N-KGFHJ-PWM6F-68KCQ-3PJBP
ServerTurbineCore=3Y8N7-3R679-4XFFH-6BTCM-X4W7X
serverazurenanoCore=9JQNQ-V8HQ6-PKB8H-GGHRY-R62H6
iotenterpriseseval=R63DV-9NPDX-QVWJF-HMR8V-M4K7D
holographic=H4MND-KF7J8-PWBTV-2P3JG-BP4FC
serverdatacenterevalCore=K4NJQ-WBJ89-PJGPP-9MTY4-R3QK2
corecountryspecific=N2434-X9D7W-8PF6X-8DV9T-8TYMD
serverstandardcorCore=B2NYV-R6JT2-H9YTQ-VQ4X3-979YK
enterprisen=WGGHN-J84D6-QYCPR-T7PJ7-X766F
professionalsinglelanguage=G3KNM-CHG6T-R36X3-9QDG6-8M8K9
serverstandardnanoCore=69NHX-WQQ7G-QVBPG-4HPQP-8XDB7
serverdatacentercor=DCXHG-67NR6-6CRMW-2PYFW-PKGD7
serverstandardCore=J768X-NXC7R-GFB9Q-BHF4H-GCVBB
gvlkcoresinglelanguage=7HNRX-D7KGG-3K4RQ-4WPJ4-YTDFH
gvlkcore=TX9XD-98N7V-6WMQ6-BX7FG-H8Q99
gvlkprofessionalworkstationn=9FNHH-K3HBT-3W4TD-6383H-6XYWF
gvlkserverazurecor=FCNV3-279Q9-BQB46-FTKXX-9HPRH
gvlkserverrdshCore=CPWHC-NT2C7-VYW78-DHDB2-PG3GK
gvlkenterprisesn=92NFX-8DJQP-P6BBQ-THF9C-7CG2H
gvlkiotenterprises=KBN8V-HFGQ4-MGXVD-347P6-PDQGT
gvlkServerDatacenterCore=D764K-2NDRG-47T6Q-P8T8W-YP6DF
gvlkServerTurbineCore=XGN3F-F394H-FD2MY-PP6FD-8MCRC
gvlkcloudeditionn=6XN7V-PCBDC-BDBRH-8DQY7-G6R44
gvlkprofessionaleducation=6TP4R-GNPTD-KYYHQ-7B7DP-J447Y
gvlkcoren=3KHY7-WNT83-DGQKR-F7HPR-844BM
gvlkprofessional=W269N-WFGWX-YVC9B-4J6C9-T83GX
gvlkprofessionaln=MH37W-N47XK-V7XM9-C7227-GCQG9
gvlkeducationn=2WH4N-8QGBV-H22JP-CT43Q-MDWWJ
gvlkeducation=NW6C2-QMPVW-D7KKK-3GKT6-VCFB2
gvlkenterprise=NPPR9-FWDCX-D2C8J-H872K-2YT43
gvlkServerTurbine=XGN3F-F394H-FD2MY-PP6FD-8MCRC
gvlkprofessionaleducationn=YVWGF-BXNMC-HTQYQ-CPQ99-66QFC
gvlkprofessionalworkstation=NRG8B-VKK3Q-CXVCJ-9G2XF-6Q84J
gvlkcloudedition=37D7F-N49CB-WQR8W-TBJ73-FM8RX
gvlkenterprises=M7XTQ-FN8P6-TTKYV-9D4CC-J462D
gvlkcorecountryspecific=PVMJN-6DFY6-9CCP6-7BKTT-D3WVR
gvlkenterprisegn=44RPN-FTY23-9VTTB-MP9BX-T84FV
gvlkenterprisen=DPH2V-TTNVB-4X9Q3-TJR4H-KHJW4
gvlkserverazurecorCore=FCNV3-279Q9-BQB46-FTKXX-9HPRH
gvlkenterpriseg=YYVX9-NTFWV-6MDM3-9PT4T-4M68B
gvlkserverrdsh=CPWHC-NT2C7-VYW78-DHDB2-PG3GK
gvlkServerDatacenter=D764K-2NDRG-47T6Q-P8T8W-YP6DF
gvlkserverstandardCore=TVRH6-WHNXV-R9WG3-9XRFY-MY832
gvlkserverstandard=TVRH6-WHNXV-R9WG3-9XRFY-MY832

[BuildInfo]

Sauce: https://forums.mydigitallife.net/threads/how-to-upgrade-from-home-sl-to-pro-or-any-other-higher-sku-and-license-switching.85294/

Find all files matching regex pattern

All PNG images starting with 5 digits followed by a dash:

$ find . -regextype posix-extended -regex "\./[[:digit:]]{5}-(.*).png

Change all file or directory permissions

$ find . -type d -exec chmod 755 {} \;
$ find . -type f -exec chmod 644 {} \;

List Root Device

$ findmnt -n -o SOURCE /

List Block Devices with Serial Numbers

$ lsblk --nodeps -o name,serial

Useful rsync Examples

  • Synchronize directories (one way):
    $ rsync -auhrP --delete --info=del,name,stats2 "/mnt/phalanx/" "/storage"
  • Copy with progress:
    $ rsync -ahr --info=progress2 some_file.mp4 /storage/

Patch Youtube App using Revanced Manager (6/22/2024)

Install Steps

Source: https://www.reddit.com/r/revancedapp/comments/1bq92r7/me_rn_trying_to_patch_youtube_revanced/

Combine find and GNU parallel

Example:

$ find . -name '*.jpg' | parallel convert -geometry 120 {} {.}_thumb.jpg
$ find . -name '*.png' | parallel --eta convert '{}' -quality 90 '{.}.jpg'
$ ls -1 *.png | parallel convert '{}' -quality 90 '{.}.jpg'
$ find . -name '*.png' -delete

Note {.} is the filename without the .jpg portion.

Setup ttyd

  • Download ttyd and put it in /opt/local/bin/ttyd:
  • Create systemd service file /etc/systemd/system/ttyd.service: 
    [Unit]                                                                                                                                                                                           
Description=TTYD
After=syslog.target
After=network.target

[Service]
ExecStart=/opt/local/bin/ttyd -W -p 9772 login
Type=simple
Restart=always
User=root
Group=root

[Install]
WantedBy=multi-user.target
    • Start and enable service:
      $ sudo systemctl start ttyd && sudo systemctl enable ttyd
  • Setup apache configuration (add to existing)
            ProxyRequests Off
        ProxyPreserveHost On
        RewriteEngine on
        RewriteCond %{HTTP:Upgrade} websocket [NC]
        RewriteCond %{HTTP:Connection} upgrade [NC]
        RewriteRule /terminal/?(.*) "ws://localhost:9772/$1" [P,L]
        ProxyPass /terminal http://localhost:9772/
        ProxyPassReverse /terminal http://localhost:9772/
    • Enable necessary apache modules:
      $ a2enmod proxy proxy_http proxy_wstunnel mod_rewrite

Increase tmux scrollback

$ cd 
$ echo "set -g history-limit 9999999" >> ~/.tmux.conf

Setup VNC

On the server side:
  1. Install x11vnc:
    $ sudo apt install x11vnc net-tools
  2. Set VNC password:
    $ x11vnc -storepasswd
  3. Start VNC service from command line:
    $ sudo x11vnc -auth guess -forever -loop -noxdamage -repeat -rfbauth /home/dsorber/.vnc/passwd -rfbport 5900 -shared
  4. To start at boot using systemd service create: /etc/systemd/system/x11vnc.service:
    [Unit]
Description=x11vnc remote desktop server
After=multi-user.target

[Service]
Type=simple
ExecStart=/usr/bin/x11vnc -auth guess -forever -loop -noxdamage -repeat -rfbauth /home/dsorber/.vnc/passwd -rfbport 5900 -shared

Restart=on-failure

[Install]
WantedBy=multi-user.target
    • Setup service
      $ sudo systemctl daemon-reload
$ sudo systemctl enable x11vnc.service 
$ sudo systemctl start x11vnc.service
On the client side:
  1. Install remmina:
    $ sudo apt install remmina
  2. Open remmina, select "VNC", and type in the server's IP address.

sshuttle

Poor man's VPN:

$ sshuttle --dns -NHr dsorber@phalanx.homelinuxserver.org:3268  192.168.0.0/16

Batch Convert PNG to JPG

NOTE: convert comes from ImageMagick

& ls -1 *.png | parallel --eta convert '{}' -quality 90 '{.}.jpg'

Crop Video with ffmpeg

$ ffmpeg -i in.mp4 -filter:v "crop=out_w:out_h:x:y" out.mp4
  • out_w is the width of the output rectangle
  • out_h is the height of the output rectangle
  • x and y specify the top left corner of the output rectangle (coordinates start at (0,0) in the top left corner of the input)

Test using ffplay:

$ ffplay -i input -vf "crop=out_w:out_h:x:y"

Build Custom Windows ISOs

Script to Convert Local manpages to html

Note use zcat if the manpages are compressed:

#!/bin/bash

mkdir -p html
for file in *; do
        if [ -f $file ] && [ "${file: -3}" != ".sh" ]; then
                echo $file
                #echo "${file%.*}" 
                pandoc --from man --to html < $file > html/${file%.*}.html
        fi
done

Display Total rsync Progress

$ rsync -a --no-i-r --info=progress2 /path/to/src /path/to/dest/

NOTE: -v does not work with --info=progress2

Source: https://serverfault.com/a/441724/472586

Slice Video File

$ ffmpeg -ss 00:01:00 -to 00:02:00  -i input.mp4 -c copy output.mp4

Explanation of the command:

  • -i: This specifies the input file. In that case, it is (input.mp4).
  • -ss: Used with -i, this seeks in the input file (input.mp4) to position.
  • 00:01:00: This is the time your trimmed video will start with.
  • -to: This specifies duration from start (00:01:40) to end (00:02:12).
  • 00:02:00: This is the time your trimmed video will end with.
  • -c copy: This is an option to trim via stream copy. (NB: Very fast)

Add new drive SW RAID5

  1. Format drive if needed (see below for steps).
  2. Add new drive to SW RAID:
    $ sudo mdadm --manage /dev/md0 --add /dev/sdd1
  3. Grow array:
    $ sudo mdadm --grow --raid-devices=4 /dev/md0

Replace failed SW RAID drive

  1. If drive is brand new be sure to create a partition on it:
    sorber@file2:~$ sudo parted /dev/sdg
GNU Parted 3.3
Using /dev/sdg
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) mklabel gpt
(parted) mkpart primary 2048s 100%                                        
(parted) print                                                            
Model: ATA ST2000DM008-2FR1 (scsi)
Disk /dev/sdg: 2000GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags: 

Number  Start   End     Size    File system  Name     Flags
 1      1049kB  2000GB  2000GB               primary

(parted) quit
  2. If needed, mark bad drive as failed:
    $ sudo mdadm --manage /dev/md0 --fail /dev/sdd
  3. Add new drive to replace failed:
    $ sudo mdadm --manage /dev/md0 --add /dev/sdd1 /dev/sdg1
  4. Improve rebuild speed slightly:
    $ sudo bash -c "echo 9999999 > /proc/sys/dev/raid/speed_limit_min" 
$ sudo bash -c "echo 9999999 > /proc/sys/dev/raid/speed_limit_max"
  5. Monitor rebuild:
    $ watch -n1 cat /proc/mdstat

Get detailed drive information

$ udevadm info --query=all --name=/dev/sda

Split flac with cue

$ sudo apt install cuetools shntool

$ shnsplit -f file.cue -t %n-%t -o flac file.flac

Fix Chrome Display Glitches

Somewhere around Chrome 86 display glitches started happening on wake. Here is how to fix:

Enabling enable-vulkan in chrome://flags then restarting the browser fixed it. If that does not work, you can also try enabling `ignore-gpu-blacklist` as well.

Setup SSH Public Key

  1. On host machine generate a key pair. This key can be reused for multiple remote machines so it's not really necessary to change the name unless multiple keys are generated, which is also not really needed.
    $ ssh-keygen -t rsa

    NOTE: for NetBeans to use/recognize the ssh key you must use the old PEM format:
    $ ssh-keygen -m PEM -t rsa
  2. Now use `ssh-copy-id` to copy the key to the remote machine (for example):
    $ ssh-copy-id -i ~/.ssh/phalanx_rsa.pub dsorber@phalanx

Read Windows Product Key from ACPI Firmware

Boot into Linux and run:

$ sudo strings /sys/firmware/acpi/tables/MSDM

Windows 10 Install and SysPrep Instructions

Very good instructions: https://theitbros.com/sysprep-windows-10-machine-guide/

Search and Replace on Binary Files

A good hex editor can do search and replace on binary files but sometimes that's not practical:

  • BBE - https://sourceforge.net/projects/bbe-/
    BBE is a binary editor with SED-like syntax:
    $ bbe -e 's/\x0a\x0d\x0a/\x0d\x0a/' infile > outfile
  • binwalk - https://github.com/ReFirmLabs/binwalk
    binwalk can search for hex strings (and do many other things)
    $ binwalk -R "\x0a\x0d\x0a" infile

WARNING: Signature '0    string    \x0a\x0d\x0a    Raw signature (\x0a\x0d\x0a)' is a self-overlapping signature!

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
242           0xF2            Raw signature (\x0a\x0d\x0a)
412           0x19C           Raw signature (\x0a\x0d\x0a)
580           0x244           Raw signature (\x0a\x0d\x0a)
749           0x2ED           Raw signature (\x0a\x0d\x0a)
. . .

Wake-on-LAN WOL

Must be enabled on interface; need MAC

$ sudo apt install etherwake
$ sudo etherwake <target MAC> -i <outgoing interface>

$ sudo etherwake b4:2e:99:a5:1e:f0 -i enp30s0

Allocator Shim

From Chromium, this is really interesting

However it is not clear how to use it outside of Chromium.

Update Geany

  1. Install prereq packages (this obviously only needs to be done once):
    $ sudo apt install libgtk-3-dev libgtkspell-dev libgtkspell3-3-dev
  2. Set compile flags:
    $ export CFLAGS="-O3 -march=native" 
$ export CXXFLAGS=$CFLAGS
  3. Build geany and set to install in /opt so it won't collide with the packaged version:
    $ tar xf geany-1.30.tar.bz2
$ cd geany-1.30/
$ ./configure --prefix=/opt/local --enable-gtk3
$ make -j20
$ sudo make install
  4. Set PKG_CONFIG_PATH environment variable so we can build the plugins.
    $ export PKG_CONFIG_PATH=/opt/local/lib/pkgconfig/
  5. Build geany-plugins:
    $ tar xf geany-plugins-1.30.tar.bz2 
$ cd geany-plugins-1.30/
$ ./configure --prefix=/opt/local --enable-gtk3
$ make -j20
$ sudo make install

Find and replace across multiple files

$ find /home/www -type f -print0 | xargs -0 sed -i 's/subdomainA\.example\.com/subdomainB.example.com/g'

Useful IPMI commands

Power measurement on the SuperMicro (probably won't work on Dell chassis as they have some proprietary version of ipmi):

sudo ipmitool -U ADMIN -P ADMIN -H 10.17.136.108 dcmi power reading

Check power status (i.e. on/off) of chassis:

sudo ipmitool -U ADMIN -P ADMIN -H 10.17.70.45 power status
sudo ipmitool -U ADMIN -P ADMIN -H 10.17.70.45 power off

Get IPMI IP address (look for "IP Address"):

sudo ipmitool lan print

Full sshfs Connect Command

sudo sshfs -o idmap=user -o uid=$(id -u) -o gid=$(id -g) -o allow_other -o follow_symlinks ryftuser@sm-X11DGQ:/home/ryftuser /mnt/sm-X11DGQ/

Side-by-side Color Diff

$ sudo apt-get install colordiff
$ colordiff -y --suppress-common-lines data.txt /ryftone/data.txt

x86 ABI

clang-format and clang-tidy

Shared Library and Related Resources

Ubuntu: List Grub Entries

gawk  'BEGIN {                                                                                                                       
  l=0                                                                                                                                
  menuindex= 0                                                                                                                       
  stack[t=0] = 0                                                                                                                     
}                                                                                                                                    

function push(x) { stack[t++] = x }                                                                                                  

function pop() { if (t > 0) { return stack[--t] } else { return "" }  }                                                              

{                                                                                                                                    

if( $0 ~ /.*menu.*{.*/ )                                                                                                             
{                                                                                                                                    
  push( $0 )                                                                                                                         
  l++;                                                                                                                               

} else if( $0 ~ /.*{.*/ )                                                                                                            
{                                                                                                                                    
  push( $0 )                                                                                                                         

} else if( $0 ~ /.*}.*/ )                                                                                                            
{                                                                                                                                    
  X = pop()                                                                                                                          
  if( X ~ /.*menu.*{.*/ )                                                                                                            
  {                                                                                                                                  
     l--;                                                                                                                            
     match( X, /^[^'\'']*'\''([^'\'']*)'\''.*$/, arr )                                                                               

     if( l == 0 )                                                                                                                    
     {                                                                                                                               
       print menuindex ": " arr[1]                                                                                                   
       menuindex++                                                                                                                   
       submenu=0                                                                                                                     
     } else                                                                                                                          
     {                                                                                                                               
       print "  " (menuindex-1) ">" submenu " " arr[1]                                                                               
       submenu++                                                                                                                     
     }                                                                                                                               
  }                                                                                                                                  
}                                                                                                                                    

}' /boot/grub/grub.cfg

h2.GCC Disable Unused Variable Warnings

#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wunused-variable" 
( your problematic library includes )
#pragma GCC diagnostic pop

See: https://stackoverflow.com/a/22708539/73878

Fix Steam after NVidia driver update

Since this has now bitten me twice... Steam requires the 32 bit version of the GL library.

$ sudo apt install libnvidia-gl-418:i386

Restart Cinnamon from TTY

  1. Cinnamon freezes
  2. Switch tty. I usually go to tty6, `Ctrl+Alt+F6`
  3. If you need to login first, do so.
  4. Type `w` (yes, just the letter) and press enter. This commands does a lot of different things, but you need it to figure out the number of the display you are using. The display number is in the column FROM. Mine is `:0` (yes, including the colon).
  5. Assuming that cinnamon is already dead (which you would notice by the windows lacking titles and that you can't move different windows around, and perhaps even not being able to use the keyboard), you type `export DISPLAY=:0; cinnamon &`, and don't forget the colon. I add the ampersand (&) only not to keep that tty busy.

Sauce: https://askubuntu.com/a/198935/392271

Ubuntu Set Specific Kernel to Boot

  • Use this command to get a list of available options:
    $ grep -Ei 'submenu|menuentry ' /boot/grub/grub.cfg | sed -re "s/(.? )'([^']+)'.*/\1 \2/" 
menuentry  Ubuntu
submenu  Advanced options for Ubuntu
    menuentry  Ubuntu, with Linux 4.4.0-34-generic
    menuentry  Ubuntu, with Linux 4.4.0-34-generic (upstart)
    menuentry  Ubuntu, with Linux 4.4.0-34-generic (recovery mode)
menuentry  System setup
  • Backup grub configuration, just in case:
    $ sudo cp /etc/default/grub /etc/default/grub.bak
  • Edit grub configuration and change `GRUB_DEFAULT` to something like this:
    $ sudo vim /etc/default/grub

GRUB_DEFAULT="Advanced options for Ubuntu>Ubuntu, with Linux 3.13.0-53-generic"
  • Update grub configuration
    $ sudo update-grub
  • See also: https://askubuntu.com/questions/216398/set-older-kernel-as-default-grub-entry

Ubuntu Networking Resources

Win10 Licensing Info

Python GTK Development Resources

cpufrequtils Governor Configuration

These instructions are intended for Ubuntu 16.04 and later (using systemd init).

  • Check the available and currently set governors. Ubuntu has an “ondemand” service that sets the governor to ondemand after boot.
    $ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors
conservative ondemand userspace powersave performance schedutil
$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor 
ondemand
  • Configure cpufrequtils to set the governor to performance at boot
    $ echo 'GOVERNOR="performance"' | sudo tee /etc/default/cpufrequtils
GOVERNOR="performance"
  • Disable the automatic ondemand service. 
    $ sudo systemctl disable ondemand
  • Reboot.
  • Now check that the governor is set to performance for every CPU:
    $ cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
performance
performance
performance
performance
…
performance
performance
performance
  • Check current CPU speeds:
    $ watch -n 1 bash -c 'cpufreq-info | grep "current CPU frequency"'

rsync via ssh example

$ rsync --info=progress2 -ahz --compress-level=9 -e "ssh -i ~/ryft-pat-f1.pem" data/ ryftuser@18.205.65.5:/ryftone/regression/pubmed/data/

Read Performance using AIO

Use this fio command (run from temp directory on disk in question) to measure max disk read performance:

$ fio --name TEST --eta-newline=5s --filename=fio-tempfile.dat --rw=read --size=10g --io_size=10g --blocksize=512k --ioengine=libaio --fsync=0 --iodepth=32 --direct=1 --numjobs=1 --runtime=300 --group_reporting

Note that DIRECT_IO is required when using AIO. (see also: https://askubuntu.com/a/991311/392271)

Remove Source Files While Creating Tarball

This is handy as it saves on disk space.

$ tar --remove-files -cf pictures.tar Pictures/

Linux Find and Remove Files

Find and remove files (even with spaces in the path) using case insensitive matching for the wildcard:

$ find . -type f -iname "*.jpg" -print0 | xargs -0 rm

Convert ASCII hex to binary
This problem seems to come up every so often. There is no need for a one-off solution:

$ xxd -r -p input.txt output.bin

https://stackoverflow.com/questions/7826526/transform-a-hex-info-to-binary-using-linux-command

Well Said

https://developers.slashdot.org/comments.pl?sid=12326098&cid=56908942

Hex Editor

I had been using [[https://hexinator.com/|Hexinator]] but I've had some minor issues with it. Today I tried [[https://www.wxhexeditor.org/home.php|wxHexEditor]] and I was pleasantly surprised.

Download from here: https://github.com/EUA/wxHexEditor

You'll need the latest version of libwxgtk<version>-dev package.

I had some minor issues with the build (which seems a somewhat bizarre arrangement...) modify the makefile to set `CC` and `CXX` directly, right about line 90.

Finally, to get it installed in the right place:

sudo make install DESTDIR=/opt/local BINDIR=bin DATADIR=share

Preprocessor Fun

To invoke the preprocessor directly:

gcc -E -DSOME_DEFINE=SOME_VALUE somefile.c

See also [[https://linux.die.net/man/1/unifdef|unifdef]]: "The unifdef utility acts on #if, #ifdef, #ifndef, #elif, #else, and #endif lines, and it understands only the commonly-used subset of the expression syntax for #if and #elif lines."

Convert Oversampled .flac to 16 bit 44.1 khz .flac

mkdir -p output && find . -type f -name "*.flac" | parallel -j16 "sox {} -b 16 -r 44100 output/{/.}.flac" 

mkdir -p output && find . -type f -name "*.flac" | parallel -j16 "sox -v 0.98 {} -b 16 -r 44100 output/{/.}.flac"

Aligned C++ STL Types

#include <vector>
#include <boost/align/aligned_allocator.hpp>

template <typename T>
using aligned_vector = std::vector<T, boost::alignment::aligned_allocator<T, 4096>>;

Command line convert .png to .jpg

$ for x in *.png; do convert $x $(basename $x .png).jpg; done

You can also add the "`-quality 90`" option if needed.

Determine CPU Family Name

$ gcc -march=native -Q --help=target|grep march

Convert .flac to .m4a (AAC)

Requires `ffmpeg` built with libfdk_aac:

$ /opt/bin/ffmpeg -i input.flac -vn -c:a libfdk_aac -vbr 3 output.m4a

All together using `find` and `parallel` for magic:

$ find . -type f -name "*.flac" | parallel -j16 "/opt/bin/ffmpeg -i {} -vn -c:a libfdk_aac -vbr 4 {/.}.m4a"

PyGObject

This is the magic needed to have stuff happen quasi asynchronously using PyGObject (GTK+):

# Yield to the main loop temporarily
while Gtk.events_pending():
    Gtk.main_iteration()

FUSE Code

*https://pastebin.com/wavKntiu

Use Windoze Schtuffs

Build Handbrake Git Build

Steps for building Handbrake using AOCC:

$ source /home/dsorber/Downloads/aocc/setenv_AOCC.sh
$ export CC=clang
$ export CXX=clang++
$ export CFLAGS="-g -O3 -mtune=native" 
$ export CXXFLAGS=$CFLAGS
$ cd ~/Downloads/HandBrake/
$ rm -rf build/
$ ./configure --prefix=/opt --enable-x265 --enable-fdk-aac --gcc=/home/dsorber/Downloads/aocc/AOCC-1.1-Compiler/bin/clang
$ cd build
$ make -j17
$ sudo make install

Regular build (18.04 and later):

$ rm -rf build/
$ ./configure --prefix=/opt/local --enable-x265 --enable-fdk-aac
$ cd build
$ make -j17
$ sudo make install

Politically Correct Linux

This guy gets it

Regarding software purist ideology vs pragmatism

"Rotate" Video Playback via Metadata

$ ffmpeg -i input.mp4 -c copy -metadata:s:v:0 rotate=90 output.mp4

Ubuntu Remove Old Kernels Oneliner

$ sudo apt-get purge $(for tag in "linux-image" "linux-headers"; do dpkg-query -W -f'${Package}\n' "$tag-[0-9]*.[0-9]*.[0-9]*" | sort -V | awk 'index($0,c){exit} //' c=$(uname -r | cut -d- -f1,2); done)

Remove New Lines (command line)

For a file:

$ tr -d '\n' < yourfile.txt

Or for output of a command (e.g.):

$ find . -type f -name '*.txt' | tr -d '\n'

Convert from .mp4 to .mkv Container

Use `avconv` to convert container without re-encoding:

$ sudo apt-get install libav-tools

$ avconv -i filme.mp4 -c copy filme.mkv

Remove String From Many File Names

On Ubuntu use Perl rename utility, e.g.:

$ rename 's/ \(1920x1080\) \[WaifuKenny\]//' *.mp4

Change Default Text Editor

I always have to Google this...

$ sudo update-alternatives --config editor

Linux View System RAM Speed

View RAM speed (actual) on Debian/Ubuntu/Mint:

$ sudo lshw -short -C memory
H/W path                      Device      Class          Description
====================================================================
/0/0                                      memory         64KiB BIOS
/0/b                                      memory         32GiB System Memory
/0/b/0                                    memory         [empty]
/0/b/1                                    memory         16GiB DIMM Synchronous 2134 MHz (0.5 ns)
/0/b/2                                    memory         [empty]
/0/b/3                                    memory         16GiB DIMM Synchronous 2134 MHz (0.5 ns)
/0/d                                      memory         768KiB L1 cache
/0/e                                      memory         4MiB L2 cache
/0/f                                      memory         16MiB L3 cache

Linux Memory Schtuff

  • From: http://unix.stackexchange.com/questions/56879/tracking-down-missing-memory-usage-in-linux

    The "memory used by a process" is not a clear cut concept in modern operating systems. What can be measured is the size of the address space of the process (SIZE) and resident set size (RSS, how many of the pages in the address space are currently in memory). Part of RSS is shared (most processes in memory share one copy of glibc, and so for assorted other shared libraries; several processes running the same executable share it, processes forked share read-only data and possibly a chunk of not-yet-modified read-write data with the parent). On the other hand, memory used for the process by the kernel isn't accounted for, like page tables, kernel buffers, and kernel stack. In the overall picture you have to account for the memory reserved for the graphics card, the kernel's use, and assorted "holes" reserved for DOS and other prehistoric systems (that isn't much, anyway).

    The only way of getting an overall picture is what the kernel reports as such. Adding up numbers with unknown overlaps and unknown left outs is a nice exercise in arithmetic, nothing more.

Reinstall grub on UEFI System

Reinstalling grub on a UEFI system is slightly different than on a traditional legacy BIOS system:

# Mount the primary drive root partition and the boot EFI partition
sudo mount /dev/sda# /mnt            # Mount root (/) partition
sudo mount /dev/sda# /mnt/boot       # Mount boot (/boot) partition (if separate from root partition; typically not)

sudo mkdir -p /mnt/boot/efi          # Create EFI partition mount point
sudo mount /dev/sda1 /mnt/boot/efi   # Mount EFI partition

# Mount your virtual filesystems:
for i in /dev /dev/pts /proc /sys /run; do sudo mount -B $i /mnt$i; done

# Chroot
sudo chroot /mnt

Now, if you have networking you'll need DNS. If using `resolvconf` (for recent versions of Ubuntu):

  1. Edit ` /etc/resolvconf/resolv.conf.d/head` and add the following line (OpenDNS server). Then save and exit:
    add nameserver 208.67.222.222
  2. Next enable updates and update:
    sudo resolvconf --enable-updates
sudo resolvconf -u
  3. Now you should have DNS. Test by pinging something.

Finally, back to fixing grub. Note that the grub install should run smoothly with no errors or warnings. If you see any something is probably wrong.

apt-get install grub-efi-amd64  # Install grub EFI bootloader (this should be all you need to do)

# manual steps if you need them for some reason
grub-install --recheck --no-floppy --force /dev/sda1  # Install grub bootloader in EFI partition

#####
echo "configfile (hd0,gpt#)/boot/grub.cfg" > /boot/efi/ubuntu/grub.cfg # Tell grub to load grub.cfg from /boot

update-grub                          # Create grub menu list
#####

exit                                 # Exit chroot

Reboot and you should be back in business.

Convert flac to VBR mp3 using ffmpeg/lame

for file in *.flac; do name=$(echo $file | sed "s/\.flac//g"); ffmpeg -i "$file" -codec:a libmp3lame -qscale:a 0 "$name".mp3; done

Create UEFI/legacy BIOS Bootable ISO

Getting rEFInd to work on Mac OS 10.11

Convert DTS 5.1 to FLAC

sudo apt-get install libdca-utils ffmpeg sound-juicer gnome-terminal easytag vlc 

for file in *.wav; do name=$(echo $file | sed "s/\.wav//g"); dcadec -o wavall "$file" > "$name"_decoded.wav; ffmpeg -i "$name"_decoded.wav "$name".flac; rm -f "$file" "$name"_decoded.wav; done

Source: http://ubuntuforums.org/archive/index.php/t-1849260.html

Windows Utils

SSH Security

https://stribika.github.io/2015/01/04/secure-secure-shell.html

Windows 10

The newest version of the KB3035583 update includes a background process which scans the 
system's Windows Registry twice a day to see if the values for the four aforementioned registry 
inputs were manually edited to disable the upgrade prompt. If they were, the process will alter 
the values, silently re-download the Windows 10 installation files (about 6 GB in total), and 
prompt the user to upgrade.

gpg-agent

These instructions are geared toward reprepro but should be useful in general for setting up `gpg-agent`.

Resolve DNS While VPN Connected

How to configure dnsmasq for DNS overrides while using a VPN connection: https://chawlasumit.wordpress.com/2014/06/15/linux-mint-openconnect-vpn-dns-issues/

Pretty Print Python Traceback

#!python
import traceback

 ...

try: 
    with open('/dev/null', 'w') as void:
        rc = not bool(subprocess.call('modprobe -r {:s}'.format(mod_name), 
                                      shell=True, stdout=void, stderr=void))
except Exception as e:
    print_failure_status()
    exc_type, exc_value, exc_traceback = sys.exc_info()
    print('\n')
    traceback.print_exception(exc_type, exc_value, exc_traceback)
    print('\n')

Generating Random Data Files for Testing

One approach is to use /dev/urandom (non-blocking pseudorandom device):

$ dd if=/dev/urandom bs=32M count=32 of=data_1GB.bin

Unfortunately /dev/urandom is a bit slow. However, you can alternatively use openssl to generate pseudorandom numbers a bit faster:
 $ openssl rand 1073741824 > data_1GB.bin
 $ for run in {1..30}; do echo $run; openssl rand 1073741824 >> data_30GB.bin; done

Cloning HDDs

When configuring multiple chassis it is necessary to clone an HDD. There are all sorts of disk imaging tools out there but it's hard to beat the trusty dd command. The pv (pipe viewer) portion isn't strictly necessary but it prints out a very useful textual status bar. The back ticked blockdev command reports the size of the drive in bytes, which pv then uses to determine overall progress. The bs argument to dd should correspond to the cache size of the drive for optimum performance (32MB is a sane default for most drives these days).

To clone a single drive to another identical drive (assuming sda is the source and sdb is the destination):

 $ sudo dd if=/dev/sda bs=32M | pv -s `sudo blockdev --getsize64 /dev/sda` | sudo dd of=/dev/sdb bs=32M

To clone to multiple drives all at once (assuming sda is the source and sdb, sdc are destinations):
 $ sudo dd if=/dev/sda bs=32M | pv -s `sudo blockdev --getsize64 /dev/sda` | tee >(sudo dd of=/dev/sdb bs=32M) | sudo dd of=/dev/sdc bs=32M

It is entirely possible to clone more drives by adding additional `>(sudo dd of=/dev/sdX bs=32M)` arguments to the tee command. Note that tee outputs to stdout so it is necessary to pipe the output of tee to the final dd command as shown above. If you don't do this then tee will spew the raw contents of the source HDD to your screen, which will make the clone take forever and look especially ugly!

For some reason RedHat/CentOS do not have the pv command or even a package for it in their repos!!! Therefore, boot from a Debian-based OS in order to clone.

Generating ISOs

 $ genisoimage -R -J -o <output.iso> <some directory>

To deal with long file names:
 $ genisoimage -R -J -l -D -joliet-long -o <output.iso> <some directory>

BIOS/grub

Creating a BIOS Boot Partition on a GPT system

$ sudo parted /dev/sda
(parted) mkpart primary 0 1024k
(parted) set 2 bios_grub on

Actiontec Router

Temp

https://www.grumpyland.com/blog/183/installing-software-raid-on-centos-567-via-ssh/

# swap is on /dev/md1
/dev/md1 none            swap    sw              0       0

# storage is on /dev/md0
/dev/md0        /storage        xfs     noatime,barrier=0 0 0
//faramir/production/sw /sw     cifs    username=samba,password=w00tw00t!,file_mode=0666,dir_mode=0777  0       0

----
Back home

Updated by 7 months ago · 40 revisions