Posts Tagged ‘match’

How to move transfer binary files encoded with base64 on Linux with Copy Paste of text ASCII encoded string

Monday, October 25th, 2021

base64-encode-decode-binary-files-to-transfer-between-servers-base64-artistic-logo

If you have to work on servers in a protected environments that are accessed via multiple VPNs, Jump hosts or Web Citrix and you have no mean to copy binary files to your computer or from your computer because you have all kind of FTP / SFTP or whatever Data Copy clients disabled on remote jump host side or CITRIX server and you still are looking for a way to copy files between your PC and the Remote server Side.
Or for example if you have 2 or more servers that are in a special Demilitarized Network Zones ( DMZ ) and the machines does not have SFTP / FTP / WebServer or other kind of copy protocol service that can be used to copy files between the hosts and you still need to copy some files between the 2 or more machines in a slow but still functional way, then you might not know of one old school hackers trick you can employee to complete the copy of files between DMZ-ed Server Host A lets say with IP address (192.168.50.5) -> Server Host B (192.168.30.7). The way to complete the binary file copy is to Encode the binary on Server Host A and then, use cat  command to display the encoded string and copy whole encoded cat command output  to your (local PC buffer from where you access the remote side via SSH via the CITRIX or Jump host.). Then decode the encoded file with an encoding tool such as base64 or uuencode. In this article, I'll show how this is done with base64 and uuencode. Base64 binary is pretty standard in most Linux / Unix OS-es today on most Linux distributions it is part of the coreutils package.
The main use of base64 encoding to encode non-text Attachment files to Electronic Mail, but for our case it fits perfectly.
Keep in mind, that this hack to copy the binary from Machine A to Machine B of course depends on the Copy / Paste buffer being enabled both on remote Jump host or Citrix from where you reach the servers as well as your own PC laptop from where you access the remote side.

base64-character-encoding-string-table

Base64 Encoding and Decoding text strings legend

The file copy process to the highly secured PCI host goes like this:
 

1. On Server Host A encode with md5sum command

[root@serverA ~]:# md5sum -b /tmp/inputbinfile-to-encode
66c4d7b03ed6df9df5305ae535e40b7d *inputbinfile-to-encode

 

As you see one good location to encode the file would be /tmp as this is a temporary home or you can use alternatively your HOME dir

but you have to be quite careful to not run out of space if you produce it anywhere 🙂

 

2. Encode the binary file with base64 encoding

 [root@serverB ~]:# base64 -w0 inputbinfile-to-encode > outputbin-file.base64

The -w0 option is given to disable line wrapping. Line wrapping is perhaps not needed if you will copy paste the data.

base64-encoded-binary-file-text-string-linux-screenshot

Base64 Encoded string chunk with line wrapping

For a complete list of possible accepted arguments check here.

3. Cat the inputbinfile-to-encode just generated to display the text encoded file in your SecureCRT / Putty / SuperPutty etc. remote ssh access client

[root@serverA ~]:# cat /tmp/inputbinfile-to-encode
f0VMRgIBAQAAAAAAAAAAAAMAPgABAAAAMGEAAAAAAABAAAAAAAAAACgXAgAAAAAAAAAAA
EAAOAALAEAAHQAcAAYAAAAEAAA ……………………………………………………………… cTD6lC+ViQfUCPn9bs

 

4. Select the cat-ted string and copy it to your PC Copy / Paste buffer


If the bin file is not few kilobytes, but few megabytes copying the file might be tricky as the string produced from cat command would be really long, so make sure the SSH client you're using is configured to have a large buffer to scroll up enough and be able to select the whole encoded string until the end of the cat command and copy it to Copy / Paste buffer.

 

5. On Server Host B paste the bas64 encoded binary inside a newly created file

Open with a text editor vim / mc or whatever is available

[root@serverB ~]:# vi inputbinfile-to-encode

Some very paranoid Linux / UNIX systems might not have even a normal text editor like 'vi' if you happen to need to copy files on such one a useful thing is to use a simple cat on the remote side to open a new File Descriptor buffer, like this:

[root@server2 ~]:# cat >> inputbinfile-to-encode <<'EOF'
Paste the string here

 

6. Decode the encoded binary with base64 cmd again

[root@serverB ~]:# base64 –decode outputbin-file.base64 > inputbinfile-to-encode

 

7. Set proper file permissions (the same as on Host A)

[root@serverB ~]:#  chmod +x inputbinfile-to-encode

 

8. Check again the binary file checksum on Host B is identical as on Host A

[root@serverB ~]:# md5sum -b inputbinfile-to-encode
66c4d7b03ed6df9df5305ae535e40b7d *inputbinfile-to-encode

As you can md5sum match on both sides so file should be OK.

 

9. Encoding and decoding files with uuencode


If you are lucky and you have uuencode installed (sharutils) package is present on remote machine to encode lets say an archived set of binary files in .tar.gz format do:

Prepare the archive of all the files you want to copy with tar on Host A:

[root@Machine1 ~]:#  tar -czvf /bin/whatever /usr/local/bin/htop /usr/local/bin/samhain /etc/hosts archived-binaries-and-configs.tar.gz

[root@Machine1 ~]:# uuencode archived-binaries-and-configs.tar.gz archived-binaries-and-configs.uu

Cat / Copy / paste the encoded content as usual to a file on Host B:

Then on Machine 2 decode:

[root@Machine2 ~]:# uuencode -c < archived-binaries-and-configs.tar.gz.uu

 

Conclusion


In this short method I've shown you a hack that is used often by script kiddies to copy over files between pwn3d machines, a method which however is very precious and useful for sysadmins like me who has to admin a paranoid secured servers that are placed in a very hard to access environments.

With the same method you can encode or decode not only binary file but also any standard input/output file content. base64 encoding is quite useful stuff to use also in bash scripts or perl where you want to have the script copy file in a plain text format . Datas are encoded and decoded to make the data transmission and storing process easier. You have to keep in mind always that Encoding and Decoding are not similar to encryption and decryption as encr. deprytion gives a special security layers to the encoded that. Encoded data can be easily revealed by decoding, so if you need to copy between the servers very sensitive data like SSL certificates Private RSA / DSA key, this command line utility tool better to be not used for sesitive data copying.

 

 

Remove string line from file on Linux and BSD – Delete entire line with string from file

Tuesday, March 15th, 2016

linux-remove-lines-containing-string-with-sed

If you're already used too using grep -v "sometring" filename to print everything from a file without the certain grepped string output and you want to do the same to delete lines based on strings without having to output the grepped string to a file and then overwritting the original file:
 

grep -v 'whatever' filename > filename1
mv filename1 filename


A much better way to delete an whole line containing a string match from a file is to use sed
sed
should be the tool of choice especially if you're scripting because sed is especially made for such batch edittings.

Here is how to do delete an entire line based on a given string:

 

sed –in-place '/some string to search and delete/d' myfilename


It might be a good idea to also create backups just to make sure something doesn't get deleted incidently to do use:

sed –in-place=.bak '/some string to search and delete/d' myfilename

If you need to wipe out an exact string from all files within a folder you might use a for loop or perl (some good examples check my previous article here)

In short to use bash's for loop here is how to backup and remove all lines with a string match within all files within a Linux directory:

 

for f in *.txt; do sed –in-place '/some string/d'
"$f"; done
find -name '*.txt' -exec sed –in-place=.bak '/some
string/d' "{}" ';'

 

BTW SED is really rich editor and some people got so much into it that there is even a sed written text (console) version of arkanoid 🙂

sed-text-editor-written-arkanoid-game-linux-bsd

If you want to break the ice and get some fun in your boring sysadmin life get sed arkanoid code from here.
I have it installed under pc-freak.net free ASCII Games entertainment service, so if you want to give it a try just login and give a try.

Enjoy 🙂

MySQL: How to check user privileges and allowed hosts to connect with mysql cli

Wednesday, April 2nd, 2014

how-to-check-user-privileges-and-allowed-hosts-to-connect-with-mysql-cli

On a project there are some issues with root admin user unable to access the server from remote host and the most probable reason was there is no access to the server from that host thus it was necessary check mysql root user privilegse and allowed hosts to connect, here SQL query to do it:
 

mysql> select * from `user` where  user like 'root%';
+——————————–+——+——————————————-+————-+————-+————-+————-+————-+———–+————-+—————+————–+———–+————+—————–+————+————+————–+————+———————–+——————+————–+—————–+——————+——————+—————-+———————+——————–+——————+————+————–+———-+————+————-+————–+—————+————-+—————–+———————-+
| Host                           | User | Password                                  | Select_priv | Insert_priv | Update_priv | Delete_priv | Create_priv | Drop_priv | Reload_priv | Shutdown_priv | Process_priv | File_priv | Grant_priv | References_priv | Index_priv | Alter_priv | Show_db_priv | Super_priv | Create_tmp_table_priv | Lock_tables_priv | Execute_priv | Repl_slave_priv | Repl_client_priv | Create_view_priv | Show_view_priv | Create_routine_priv | Alter_routine_priv | Create_user_priv | Event_priv | Trigger_priv | ssl_type | ssl_cipher | x509_issuer | x509_subject | max_questions | max_updates | max_connections | max_user_connections |
+——————————–+——+——————————————-+————-+————-+————-+————-+————-+———–+————-+—————+————–+———–+————+—————–+————+————+————–+————+———————–+——————+————–+—————–+——————+——————+—————-+———————+——————–+——————+————+————–+———-+————+————-+————–+—————+————-+—————–+———————-+
| localhost                      | root | *5A07790DCF43AC89820F93CAF7B03DE3F43A10D9 | Y           | Y           | Y           | Y           | Y           | Y         | Y           | Y             | Y            | Y         | Y          | Y               | Y          | Y          | Y            | Y          | Y                     | Y                | Y            | Y               | Y                | Y                | Y              | Y                   | Y                  | Y                | Y          | Y            |          |            |             |              |             0 |           0 |               0 |                    0 |
| server737                        | root | *5A07790DCF43AC89820F93CAF7B03DE3F43A10D9 | Y           | Y           | Y           | Y           | Y           | Y         | Y           | Y             | Y            | Y         | Y          | Y               | Y          | Y          | Y            | Y          | Y                     | Y                | Y            | Y               | Y                | Y                | Y              | Y                   | Y                  | Y                | Y          | Y            |          |            |             |              |             0 |           0 |               0 |                    0 |
| 127.0.0.1                      | root | *5A07790DCF43AC89820F93CAF7B03DE3F43A10D9 | Y           | Y           | Y           | Y           | Y           | Y         | Y           | Y             | Y            | Y         | Y          | Y               | Y          | Y          | Y            | Y          | Y                     | Y                | Y            | Y               | Y                | Y                | Y              | Y                   | Y                  | Y                | Y          | Y            |          |            |             |              |             0 |           0 |               0 |                    0 |
| server737.server.myhost.net | root | *5A07790DCF43FC89820A93CAF7B03DE3F43A10D9 | Y           | Y           | Y           | Y           | Y           | Y         | Y           | Y             | Y            | Y         | Y          | Y               | Y          | Y          | Y            | Y          | Y                     | Y                | Y            | Y               | Y                | Y                | Y              | Y                   | Y                  | Y                | Y          | Y            |          |            |             |              |             0 |           0 |               0 |                    0 |
| server4586                        | root | *5A07790DCF43AC89820F93CAF7B03DE3F43A10D9 | N           | N           | N           | N           | N           | N         | N           | N             | N            | N         | N          | N               | N          | N          | N            | N          | N                     | N                | N            | N               | N                | N                | N              | N                   | N                  | N                | N          | N            |          |            |             |              |             0 |           0 |               0 |                    0 |
| server4586.myhost.net              | root | *5A07790DCF43AC89820F93CAF7B03DE3F43A10D9 | N           | N           | N           | N           | N           | N         | N           | N             | N            | N         | N          | N               | N          | N          | N            | N          | N                     | N                | N            | N               | N                | N                | N              | N                   | N                  | N                | N          | N            |          |            |             |              |             0 |           0 |               0 |                    0 |
+——————————–+——+——————————————-+————-+————-+————-+————-+————-+———–+————-+—————+————–+———–+————+—————–+————+————+————–+————+———————–+——————+————–+—————–+——————+——————+—————-+———————+——————–+——————+————+————–+———-+————+————-+————–+—————+————-+—————–+———————-+
6 rows in set (0.00 sec)

mysql> exit


Here is query explained:

select * from `user` where  user like 'root%'; query means:

select * – show all
from `user` – from user database
where user like 'root%' – where there is match in user column to any string starting with 'root*',
 

find text strings recursively in Linux and UNIX – find grep in sub-directories command examples

Tuesday, May 13th, 2014

unix_Linux_recursive_file_search_string_grep
GNU Grep
is equipped with a special option "-r" to grep recursively. Looking for string in a file in a sub-directories tree with the -r option is a piece of cake. You just do:

grep -r 'string' /directory/

or if you want to search recursively non-case sensitive for text

grep -ri 'string' .
 

Another classic GNU grep use (I use almost daily) is whether you want to match all files containing (case insensitive) string  among all files:

grep -rli 'string' directory-name
 

Now if you want to grep whether a string is contained in a file or group of files in directory recursively on some other UNIX like HP-UX or Sun OS / Solaris where there is no GNU grep installed by default here is how to it:

find /directory -exec grep 'searched string' {} dev/null ;

Note that this approach to look for files containing string on UNIX is very slowThus on not too archaic UNIX systems for some better search performance it is better to use xargs;

find . | xargs grep searched-string


A small note to open here is by using xargs there might be weird results when run on filesystems with filenames starting with "-".

Thus comes the classical (ultimate) way to grep for files containing string with find + grep, e.g.

find / -exec grep grepped-string {} dev/null ;

Another way to search a string recursively in files is by using UNIX OS '*' (star) expression:

grep pattern * */* */*/* 2>/dev/null

Talking about recursive directory text search in UNIX, should mention  another good GNU GREP alternative ACK – check it on betterthangrep.com 🙂 . Ack is perfect for programmers who have to dig through large directory trees of code for certain variables, functions, objects etc.

 

10 must know and extremely useful Linux commands that every sys admin should know

Tuesday, July 30th, 2013

10 must know extremely useful gnu linux command line tools tips and tricks
There are plenty of precious command line stuff every admin should be aware on Linux. In this article I just decided to place some I use often and are interesting to know. Below commands are nothing special and probably many of experienced sys admins already know them. However I'm pretty sure novice admins and start-up Linux enthusiasts will find it useful. I know there much more to be said on the topic. So anyone is mostly welcome to share his used cmds.
 
1. Delete all files in directory except files with certain file extension

It is good trick to delete all files in directory except certain file formats, to do so:

root@linux:~# rm !(*.c|*.py|*.txt|*.mp3)

2. Write command output to multiple files (tee)

The normal way to write to file is by using redirect (to overwrite file) ">" or (to append to file) ">>";. However when you need to write output to multiple files there is a command called tee, i.e.:

root@linux:~# ps axuwwf | tee file1 file2 file3

3. Search for text in plain text file printing number of lines after match

Whether you need to print all number of lines after match of "search_text" use:

root@linux:~# grep -A 5 -i "search_text" text_file.txt

4. Show all files where text string is matched with GREP (Search for text recursively)

Searching for text match is extremely helpful for system administration. I use  grep recursive (capability) almost on daily basis:

root@websrv:/etc/dovecot# grep -rli text *
conf.d/10-auth.conf
conf.d/10-mail.conf
dovecot.conf

-l (instructs to only print file names matching string), -r (stands for recursive search), and -i flag (instructs grep to print all matches  inogoring case-sensitivity ( look for text nomatter if with capital or small letters)

5. Finding files and running command on each file type matched

In Linux with find command it is possible to search for files and run command on each file matched.
Lets say you we want to look in current directory for all files .swp (temporary) files produced so often by VIM and wipe them out:

root@linux:~# find . -iname '*.swp*' -exec rm -f {} \;

6. Convert DOS end of file (EOF) to UNIX with sed

If it happens you not have dos2unix command installed on Linux shell and you need to translate DOS end of file (\r\n – return carriage, new line) to UNIX's (\r – return carriage)), do it with sed:

root@linux:~# sed 's/.$//' filename

7. Remove file duplicate lines with awk:

cat test.txt
test
test
test duplicate
The brown fox jump over ...
Richard Stallman rox

root@linux:~# awk '!($0 in array) { array[$0]; print }' test.txt
test
test duplicate
The brown fox jump over ...
Richard Stallman rox

To remove duplicate text from all files in directory same can be easily scripped with bash for loop:

root@linux:~# for i in *; do
awk '!($0 in array) { array[$0]; print }' $i;
done

8. Print only selected columns from text file

To print text only in 1st and 7th column in plain text file with awk:

root@linux:~# awk '{print $1,$6;}' filename.txt ...

To print only all existing users on Linux with their respective set shell type:

root@linux:~# cat /etc/passwd|sed -e 's#:# #g'|awk '{print $1,$6;}'

9. Open file with VIM text editor starting from line

I use only vim for console text processing, and I often had to edit and fix file which fail to compile on certain line number. Thus use vim to open file for writing from necessary line num. To open file and set cursor to line 35 root@linux:~# vim +35 /home/hipo/current.c

10. Run last command with "!!" bash shorcut

Lets say last command you run is uname -a:

root@websrv:/home/student# uname -a
Linux websrv 3.2.0-4-686-pae #1 SMP Debian 3.2.46-1 i686 GNU/Linux

To re-run it simply type "!!":

root@websrv:/home/student# !!
uname -a
Linux websrv 3.2.0-4-686-pae #1 SMP Debian 3.2.46-1 i686 GNU/Linux

root@websrv:/home/student#

 

6 days in sickness

Friday, August 10th, 2007

My physical health was quite not good during the last 6 / 7 days. Today it was a quiet day.I haven’t prayed seriously for few days but I can’t. Since my life looks like going nowhere.There is almost nothing in this town which keeps me still. I went to the Old Dobrich inMino’s coffee. But after a little argue and being a little rude to a girl I leavedthis awful mess. This guys are not a good company/match for me. It seems I don’t have friendsexcept Lily. Well I hope at least I haven’t builded all the time for nothing.Thanks Goodness that at least at work there isn’t a lot of work so I’m in a period of recovery.The world is going mad. I’m starting to scare my self. Seems like, life is created to be livednot to think about it’s purpose.END—–