Archive for October, 2024

How to install and use WSL 2 Windows native Linux emulation Debian and Ubuntu Linux on Windows 10 / Windows 11

Thursday, October 31st, 2024

start-with-wsl-windows-emulation-linux-install-and-use-easily-linux-and-windows-together-with-no-external-software-tux-penguin-logo

WSL (Windows Subsystem for Linux) is perhaps relatively rarely known to the old school sys admins rats who usually use stuff like QEMU / KVM for Windows or Virtualbox / VMWare for Host machine.
However most people most lileky heard but never used or heard about the native (container like) virtualization WSL which was introduced in Windows 10 and Windows 11  as an attempt from Microsoft to improve the interoperability between Windows and Linux.
WSL version 1 and ver 2 allows Microsoft Windows for using a Linux environment without the need for a separate virtual machine.

In Windows 10, it is existing in Windows 10 Professional version can be installed either by joining the Windows Insider program or manually via Microsoft Store or Winget.
Hence perhaps you don't know that WSL virtualization can be used by those who want to mix Linux and Windows or for example get an advantages against dual-boot (installing Linux and Windows on the same computer).
Even better most significant WSL pros is you can literally running both systems at the same time without the need to run or stop every software that’s running and reboot to another system.

Procedure to set up a WSL is simple and similar to setting up a real Linux OS, therefore this guide can also be used as a reference to Linux setup.The specifications of WSL setup procedure are mainly in Install WSL and then setup any packages you would like to use for example if you want to be able to access remotely the WSL emulated Debian / Ubuntu or other of the installable distros via OpenSSH server.

1. Requirements to install and use WSL Linux emulation

To have the wsl subsystem used on Windows 10 or Windows 11 requirements:

You must be running Windows 10 version 2004 and higher (Build 19041 and higher) or Windows 11 to use the commands below. If you are on earlier versions please see the manual install page.

2. List available installable Linux distributions
 

WSL subsystem has ported only a certain set of Linux distributions, so if you need a very specific and unique Linux distribution, you would perhaps need to use Hyper-V virtualization or Virtualbox / VMWare.
However for people like me who are mainly using Debian GNU / Linux on daily basis as well as some OracleLinux admins / SUSE it is a perfect solution.

PS C:\Windows\System32\WindowsPowerShell\v1.0> wsl –list –online
The following is a list of valid distributions that can be installed.
Install using 'wsl.exe –install <Distro>'.

NAME                            FRIENDLY NAME
Ubuntu                          Ubuntu
Debian                          Debian GNU/Linux
kali-linux                      Kali Linux Rolling
Ubuntu-18.04                    Ubuntu 18.04 LTS
Ubuntu-20.04                    Ubuntu 20.04 LTS
Ubuntu-22.04                    Ubuntu 22.04 LTS
Ubuntu-24.04                    Ubuntu 24.04 LTS
OracleLinux_7_9                 Oracle Linux 7.9
OracleLinux_8_7                 Oracle Linux 8.7
OracleLinux_9_1                 Oracle Linux 9.1
openSUSE-Leap-15.6              openSUSE Leap 15.6
SUSE-Linux-Enterprise-15-SP5    SUSE Linux Enterprise 15 SP5
SUSE-Linux-Enterprise-15-SP6    SUSE Linux Enterprise 15 SP6
openSUSE-Tumbleweed             openSUSE Tumbleweed


 

3. Install Linux distribution for a first time

PS C:\Windows\System32\WindowsPowerShell\v1.0> wsl –install

wsl2-windows-virtualization-install-virtual-machine-debian1

The default Linux distribution that will get installed inside WLS Virtlualization is Ubuntu.

4. Install Debian GNU / Linux distribution as a second distro

 

PS C:\Windows\System32\WindowsPowerShell\v1.0> wsl –install Debian

windows-wsl-linux-emulation/wsl2-windows-virtualization-install-virtual-machine-debian3

That second installed distro would make Debian now the default one to boot by WSL.

To run the fresh installed Debian GNU / Linux distribution, run only wsl command with no arguments.

# wsl

 

PS C:\Windows\System32\WindowsPowerShell\v1.0> wsl –set-version Debian 2
For information on key differences with WSL 2 please visit https://aka.ms/wsl2
Conversion in progress, this may take a few minutes.
The distribution is already the requested version.
Error code: Wsl/Service/WSL_E_VM_MODE_INVALID_STATE
PS C:\Windows\System32\WindowsPowerShell\v1.0> wsl –set-version 2
There is no distribution with the supplied name.
Error code: Wsl/Service/WSL_E_DISTRO_NOT_FOUND
PS C:\Windows\System32\WindowsPowerShell\v1.0>

Simply pressting CTRL + D from the actively running WSL emulated Linux (that is pretty much like a native Windows docker container if we have to compare to Linux) would stop the VM.
 

5. List runnable / installed VM Linux distributions
 

To list the available runnable Linux VMs on your Windows  status on Windows Subsystem for Linux:

PS C:\Windows\System32\WindowsPowerShell\v1.0> wsl –list –verbose
  NAME      STATE           VERSION
* Debian    Stopped         2
  Ubuntu    Stopped         2

PS C:\Windows\System32\WindowsPowerShell\v1.0>


6. Run and check recent installed Linux distribution version

wsl2-windows-virtualization-install-virtual-machine-debian4

To run the newly install Debian Virtualized Linux (which as you can see is the default set distribution to run by WSL virtualization) simply type 

PS C:\Windows\System32\WindowsPowerShell\v1.0> wsl

hipo@PC2LP3:/mnt/c/Windows/System32/WindowsPowerShell/v1.0$hipo@WL-2SLPWL3:/mnt/c/Windows/System32/WindowsPowerShell/v1.0$ cd ~
hipo@PC2LP3:~$

 

hipo@PC2LP3:~$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
hipo@WL-2SLPWL3:~$

 

7. Update the Debian distribuion packages to latest available

hipo@PC2LP3:~$ sudo su – root
hipo@PC2LP3:~# apt update –fix-missing


8. Install openssh server to be able to connect to the WSL hosted Virtual Machine

hipo@PC2LP3:/home/hipo# apt install openssh-server –yes


windows-wsl-linux-emulation

 

root@PC2LP3:/home/hipo# systemctl start openssh-server telnet
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down
root@WL-2SLPWL3:/home/hipo# /etc/init.d/ssh start
Starting OpenBSD Secure Shell server: sshd.
root@WL-2SLPWL3:/home/hipo# ps -ef|grep -i ssh
root        30     9  0 18:19 ?        00:00:00 sshd: /usr/sbin/sshd [listener] 0 of 10-100 startups
root        32    15  0 18:20 pts/1    00:00:00 grep -i ssh

 

windows-wsl-linux-emulation

By default a fresh new installed VM would have a process list like below:

root@PC2LP3:/home/hipo# ps axuwef


wsl2-windows-virtualization-install-virtual-machine-debian7

To be able to have ifconfig and a number of other network tools it is useful to install net-tools package

root@PC2LP3:/home/hipo# apt install net-tools –yes

root@PC2LP3:/home/hipo# /sbin/ifconfig

Once the WSL VM and OpenSSHD is run you can try to telnet or ssh to the VM locally or remotely.

root@PC2LP3:/home/hipo# telnet localhost 22
Trying 127.0.0.1…
Connected to localhost.
Escape character is '^]'.
SSH-2.0-OpenSSH_9.2p1 Debian-2+deb12u3

9. Run commands directly from Windows command line or Powershell
 

You can also use the powershell to run commands via the virtualized Linux environment using simple syntax

# wsl [cmd-to-run]

PS C:\Windows\System32\WindowsPowerShell\v1.0> wsl ls /
bin   dev  home  lib    lost+found  mnt  proc  run   srv  tmp  var
boot  etc  init  lib64  media       opt  root  sbin  sys  usr
PS C:\Windows\System32\WindowsPowerShell\v1.0> wsl ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 18:07 hvc0     00:00:00 /init
root         5     1  0 18:07 hvc0     00:00:00 plan9 –control-socket 5 –log-level 4 –server-fd 6 –pipe-fd 8 –log-t
root         8     1  0 18:07 ?        00:00:00 /init
root         9     8  0 18:07 ?        00:00:00 /init
hipo        10     9  0 18:07 pts/0    00:00:00 ps -ef

PS C:\Windows\System32\WindowsPowerShell\v1.0>

10. Enable systemd on Linux distribution in WSL 2

Once you boot into the WSL installed distro shell edit /etc/wsl.conf:

$ vim /etc/wsl.conf

[boot]
systemd=true

11. Setting extra useful variables to boot the WSL emulated Linux VM 
 

root@debian-wsl:/home/hipo# cat /etc/wsl.conf
[boot]
systemd=true

# Automatically mount Windows drive when the distribution is launched
[automount]

# Set to true will automount fixed drives (C:/ or D:/) with DrvFs under the root directory set above. Set to false means drives won't be mounted automatically, but need to be mounted manually or with fstab.
enabled = true

# Sets the directory where fixed drives will be automatically mounted. This example changes the mount location, so your C-drive would be /c, rather than the default /mnt/c.
root = /

# DrvFs-specific options can be specified.
options = "metadata,uid=1003,gid=1003,umask=077,fmask=11,case=off"

# Sets the `/etc/fstab` file to be processed when a WSL distribution is launched.
mountFsTab = true

# Network host settings that enable the DNS server used by WSL 2. This example changes the hostname, sets generateHosts to false, preventing WSL from the default behavior of auto-generating /etc/hosts, and sets generateResolvConf to false, preventing WSL from auto-generating /etc/resolv.conf, so that you can create your own (ie. nameserver 1.1.1.1).
[network]
hostname = debian-wsl
generateHosts = true
generateResolvConf = true

# Set whether WSL supports interop processes like launching Windows apps and adding path variables. Setting these to false will block the launch of Windows processes and block adding $PATH environment variables.
[interop]
enabled = false
appendWindowsPath = false

# Set the user when launching a distribution with WSL.
[user]
default = hipo

# Set a command to run when a new WSL instance launches. This example starts the Docker container service.
#[boot]
#command = service docker start

root@debian-wsl:/home/hipo#

To learn about on Advanced settings configuration in WSL check out official Microsoft documentation here

12. Shutting down a running emulated Linux VM

If you have run a WSL VM and you want to shut it down do:

# wsl shutdown


If you at a point want to delete / uninstall the installed distribution you can do

# wsl –terminate Distro_Name
# wsl –uninstall Distro_Name


Or you if you want to do a cleanup of the stored files inside the installed distribution (if you have stored files), do:

# wsl –unregister Distro_Name


For more in depth details check out the manual
 

PS C:\Windows\System32\WindowsPowerShell\v1.0> wsl –help
Copyright (c) Microsoft Corporation. All rights reserved.
For privacy information about this product please visit https://aka.ms/privacy.

Usage: wsl.exe [Argument] [Options…] [CommandLine]

Arguments for running Linux binaries:

    If no command line is provided, wsl.exe launches the default shell.

    –exec, -e <CommandLine>
        Execute the specified command without using the default Linux shell.

    –shell-type <standard|login|none>
        Execute the specified command with the provided shell type.

    —
        Pass the remaining command line as-is.

Options:
    –cd <Directory>
        Sets the specified directory as the current working directory.
        If ~ is used the Linux user's home path will be used. If the path begins
        with a / character, it will be interpreted as an absolute Linux path.
        Otherwise, the value must be an absolute Windows path.

    –distribution, -d <Distro>
        Run the specified distribution.

    –user, -u <UserName>
        Run as the specified user.

    –system
        Launches a shell for the system distribution.

Arguments for managing Windows Subsystem for Linux:

    –help
        Display usage information.

    –debug-shell
        Open a WSL2 debug shell for diagnostics purposes.

    –install [Distro] [Options…]
        Install a Windows Subsystem for Linux distribution.
        For a list of valid distributions, use 'wsl.exe –list –online'.

        Options:
            –no-launch, -n
                Do not launch the distribution after install.

            –web-download
                Download the distribution from the internet instead of the Microsoft Store.

            –no-distribution
                Only install the required optional components, does not install a distribution.

            –enable-wsl1
                Enable WSL1 support.

    –manage <Distro> <Options…>
        Changes distro specific options.

        Options:
            –move <Location>
                Move the distribution to a new location.

            –set-sparse, -s <true|false>
                Set the vhdx of distro to be sparse, allowing disk space to be automatically reclaimed.

    –mount <Disk>
        Attaches and mounts a physical or virtual disk in all WSL 2 distributions.

        Options:
            –vhd
                Specifies that <Disk> refers to a virtual hard disk.

            –bare
                Attach the disk to WSL2, but don't mount it.

            –name <Name>
                Mount the disk using a custom name for the mountpoint.

            –type <Type>
                Filesystem to use when mounting a disk, if not specified defaults to ext4.

            –options <Options>
                Additional mount options.

            –partition <Index>
                Index of the partition to mount, if not specified defaults to the whole disk.

    –set-default-version <Version>
        Changes the default install version for new distributions.

    –shutdown
        Immediately terminates all running distributions and the WSL 2
        lightweight utility virtual machine.

    –status
        Show the status of Windows Subsystem for Linux.

    –unmount [Disk]
        Unmounts and detaches a disk from all WSL2 distributions.
        Unmounts and detaches all disks if called without argument.

    –uninstall
        Uninstalls the Windows Subsystem for Linux package from this machine.

    –update
        Update the Windows Subsystem for Linux package.

        Options:
            –pre-release
                Download a pre-release version if available.

    –version, -v
        Display version information.

Arguments for managing distributions in Windows Subsystem for Linux:

    –export <Distro> <FileName> [Options]
        Exports the distribution to a tar file.
        The filename can be – for stdout.

        Options:
            –vhd
                Specifies that the distribution should be exported as a .vhdx file.

    –import <Distro> <InstallLocation> <FileName> [Options]
        Imports the specified tar file as a new distribution.
        The filename can be – for stdin.

        Options:
            –version <Version>
                Specifies the version to use for the new distribution.

            –vhd
                Specifies that the provided file is a .vhdx file, not a tar file.
                This operation makes a copy of the .vhdx file at the specified install location.

    –import-in-place <Distro> <FileName>
        Imports the specified .vhdx file as a new distribution.
        This virtual hard disk must be formatted with the ext4 filesystem type.

    –list, -l [Options]
        Lists distributions.

        Options:
            –all
                List all distributions, including distributions that are
                currently being installed or uninstalled.

            –running
                List only distributions that are currently running.

            –quiet, -q
                Only show distribution names.

            –verbose, -v
                Show detailed information about all distributions.

            –online, -o
                Displays a list of available distributions for install with 'wsl.exe –install'.

    –set-default, -s <Distro>
        Sets the distribution as the default.

    –set-version <Distro> <Version>
        Changes the version of the specified distribution.

    –terminate, -t <Distro>
        Terminates the specified distribution.

    –unregister <Distro>
        Unregisters the distribution and deletes the root filesystem.
PS C:\Windows\System32\WindowsPowerShell\v1.0>

Once wsl is installed you can run it directly from Windows start menu, by searching for the name of the distribution you would like to run for example to run my Debian WSL running emulator::

Sum it up

What was shown up is how to run in parallel virtualized Linux distribution on Windows 10 and Windows 11 and how to install update to latest and run opensshd server to be able to ssh into the WSL Linux virtual machine remotely.
.Also i've shown you, How to test ssh is reachable and how to stop / start or destroy and cleanup any stored files for  VM if necessery, as well as how to apply some extra advanced configurations to boot VM for.

Using WSL is not the best virtualization ever but anyways it is an alternative for people employed in Domain attached Windows PCs part of Big Corporations, where VirtualBox use is blocked / prohibited and you still need to experiment or develop Shell scripts or software on Python / Perl / Ruby on Linux before you  do stuff on the PreProd or Production Linux host.

That's all folks, Enjoy ! 🙂
 

How to split large files in Windows via split command line and File Archive GUI tool easily

Tuesday, October 22nd, 2024

Moving around a very large files especially Virtualbox Virtual Machines or other VM formats between Windows host and OneDrive might be a problem due to either Azure Cloud configured limitations, or other reasons that your company Domain Administrator has configured, thus if you have to migrate your old Hardware Laptop PC Windows 10 to a newer faster better Harware / Better Performance Notebook Computer with Windows 11 and you still want to keep and move your old large files in this short and trivial article, will explain how.

The topic is easily and most of novice sysadmins should have already be faced to bump into something like this but anyways i found useful to mention about Git for Windows, as it is really useful too thus wrote this small article.

The moved huge files, in my case an experimental Virtual Machines Images which I needed to somehow migrate on the new Freshly installed Windows laptop, the Large files were 40 / 80 etc. Gigabytes or whatever large amount of files from your PC to the Cloud Onedrive and of course the most straight forward thing i tried was to simply add the file for inclusion into the Onedrive storage (via OneDrive tool setup interface), however this file, failed due to OneDrive Cloud file format security limitations or Antivirus solutions configured to filter out the large file copying or even a prohibition to be able to include any kind of Virtual Machines ISOs straight into the cloud.

With this big files comes the question:

How to copy the Virtual Machines from your Old Hardware Laptop to the Cloud (without being able to use an external SSD Hard Drive or a USB SSD Flash drive, due to Domain policy configured for your windows to be unable to copy to externally connected Drive but only to read from such.) ?
 

Here are few sample approaches to do it both from command line (useful if you have to repeat the process or script it and deploy to multiple hosts) or for single hosts via an Archiver tool:

 

1. Using split command Git for Windows (Bash) MINGW64 shell 

Download Git for Windows – https://git-scm.com/download install it and you will get the MINGW64 bash for Windows executable.

Run it either invoke bash command from command line or trigger Windows Run command prompt (Windows button + R) and type full path to executable
 

C:\Program Files\Git\git-bash.exe


Git-for-Windows-bash-for-windows-MINGW64-windows-11-screenshot

Use the integrated program split and to cut it into pieces use:

 

# split MyVeryLargeFileVM.vdi -b 800m


To split the .VDI virtualbox file to lets say 5 Gigabite pieces:

# split MyVeryLargeFile.vdi -b 5g

The output files will be named pieces will be named as in a normal UNIX / GNU split command in the format and each piece of 5GB will be named like:

xaa
xab
xac
xad

If you want to get a more meaningful name for the spilitted files you can set a generated split file prefix with suffixes to be 5 digits long:

# split MyVeryLargeFile.vdi MyVeryLargeFileVM-parts_ -b 5g -d -a 5

  • the -d flag for using numerical suffixes (instead of default aa, ab, ac, etc…),
  • and the option -a 5 to tell it I want the suffixes to be 5 digits long:

2. Split large files by Archiving them with Winrar (ShareWare) tool

If you have already Winrar installed and you don't want to bother with too much typing from the command line, You can use good old WinRAR as a file splitter/joiner as well.

To split a file into smaller files, select "Store" as the compression method and enter the desired value (bytes) into "Split to volumes" box.
This way you can have split files named as filename.part1.rar, filename.part2.rar, etc.

WinRAR_cut-split-large-files-into-pieces-screenshot-Windows

3. Split files with 7-Zip (FreeWare)

Assuming you have the 7-Zip installed on the PC, you can do the archiving of the Big file to a smaller pieces one, you can create the splitted file from 7Zip interfaces menus:

7zip-file-split-files-into-multiple-pieces-windows-screenshot

Or directly cut the single file into multiple volumes, directly from Windows Explorer by Selecting the file and using fall down menus :
7zip-creation-of-multiple-parts-file-from-single-one-screenshot-Windows

7-zip-split-huge-files-to-lower-parts-set-volume-size

Sum it up what learned ? 

What we learned is how to cut large files into multiple single consequential ones for easy copy between Network sides, via both Git 4 Windows and manual copy paste of parted multiple files to OneDrive / DropBox / pCloud or Google Drive.
There is a plenty of other approaches to take as there is also file GUI tools, besides using GNU Win / Gnu Tools for Windows or Cygwin / Gsplit GUI tool  or some kind of the many Archiver toolsavailable for Windows, another option to split the large files is to use a bunch of PowerShell and Batch scripts written that can help you do the file split for both binaries files or Text files. but i'll stop here as I believe that is pretty much enough for most basic needs.

 

How to Copy / Backup Windows USB drive from one USB to a second

Friday, October 18th, 2024

Did you know that when you copy all the files from a USB Drive you don’t copy all the data?

Did you know that there may be files that are not even visible?

In this tutorial you will discover how to copy all of your USB Drive sector by sector, that is to say, that you will see how to create a copy identical to your USB drive without missing anything!

This can be useful if you have formatted your USB stick in error and want to use it, you can create an image for the USB Drive on your computer and then you can recover the formatted data in the image afterward!

The software used in this tutorial is called ImageUSB, it is free, portable, and easy to use.

Don’t use this method if you want only to copy some files, use this to clone/backup your USB Drive with all its master boot record, partition tables, and data.

Let’s go!

Clone Your USB Drive with ImageUSB on Windows 10

Start by downloading and extracting ImageUSB from this official URL: https://www.osforensics.com/tools/write-usb-images.html

Double-click on  imageUSB.exe .

Select your USB Drive from the list, select “Create image from USB drive“. Choose the location for the binary image file (.bin) that will be created from the USB drive.

Click on “Create“.Click “Yes” to confirm your choices.

imageusb clone usb flash drive backup restore 3 create image

Click “Yes” to overwrite the bin file in case it’s already there.

Wait for a couple of minutes…

After the image is created you should see this message. Click “OK“.

Now if you want to restore an image to your USB Drive, just select your USB Drive and choose “Write image to USB drive“. Choose your bin image and click on “Write“.

imageusb clone usb flash drive backup restore 7 write

This program is not recommended on different sizes USB Drives…
Use it mostly for backup/restore on the same USB Drive for your bootable software.

There you have it, the copy of USB to second USB completed !

Enjoy !