Posts Tagged ‘plain’

How to move transfer binary files encoded with base64 on Linux with Copy Paste of text ASCII encoded string

Monday, October 25th, 2021

base64-encode-decode-binary-files-to-transfer-between-servers-base64-artistic-logo

If you have to work on servers in a protected environments that are accessed via multiple VPNs, Jump hosts or Web Citrix and you have no mean to copy binary files to your computer or from your computer because you have all kind of FTP / SFTP or whatever Data Copy clients disabled on remote jump host side or CITRIX server and you still are looking for a way to copy files between your PC and the Remote server Side.
Or for example if you have 2 or more servers that are in a special Demilitarized Network Zones ( DMZ ) and the machines does not have SFTP / FTP / WebServer or other kind of copy protocol service that can be used to copy files between the hosts and you still need to copy some files between the 2 or more machines in a slow but still functional way, then you might not know of one old school hackers trick you can employee to complete the copy of files between DMZ-ed Server Host A lets say with IP address (192.168.50.5) -> Server Host B (192.168.30.7). The way to complete the binary file copy is to Encode the binary on Server Host A and then, use cat  command to display the encoded string and copy whole encoded cat command output  to your (local PC buffer from where you access the remote side via SSH via the CITRIX or Jump host.). Then decode the encoded file with an encoding tool such as base64 or uuencode. In this article, I'll show how this is done with base64 and uuencode. Base64 binary is pretty standard in most Linux / Unix OS-es today on most Linux distributions it is part of the coreutils package.
The main use of base64 encoding to encode non-text Attachment files to Electronic Mail, but for our case it fits perfectly.
Keep in mind, that this hack to copy the binary from Machine A to Machine B of course depends on the Copy / Paste buffer being enabled both on remote Jump host or Citrix from where you reach the servers as well as your own PC laptop from where you access the remote side.

base64-character-encoding-string-table

Base64 Encoding and Decoding text strings legend

The file copy process to the highly secured PCI host goes like this:
 

1. On Server Host A encode with md5sum command

[root@serverA ~]:# md5sum -b /tmp/inputbinfile-to-encode
66c4d7b03ed6df9df5305ae535e40b7d *inputbinfile-to-encode

 

As you see one good location to encode the file would be /tmp as this is a temporary home or you can use alternatively your HOME dir

but you have to be quite careful to not run out of space if you produce it anywhere 🙂

 

2. Encode the binary file with base64 encoding

 [root@serverB ~]:# base64 -w0 inputbinfile-to-encode > outputbin-file.base64

The -w0 option is given to disable line wrapping. Line wrapping is perhaps not needed if you will copy paste the data.

base64-encoded-binary-file-text-string-linux-screenshot

Base64 Encoded string chunk with line wrapping

For a complete list of possible accepted arguments check here.

3. Cat the inputbinfile-to-encode just generated to display the text encoded file in your SecureCRT / Putty / SuperPutty etc. remote ssh access client

[root@serverA ~]:# cat /tmp/inputbinfile-to-encode
f0VMRgIBAQAAAAAAAAAAAAMAPgABAAAAMGEAAAAAAABAAAAAAAAAACgXAgAAAAAAAAAAA
EAAOAALAEAAHQAcAAYAAAAEAAA ……………………………………………………………… cTD6lC+ViQfUCPn9bs

 

4. Select the cat-ted string and copy it to your PC Copy / Paste buffer


If the bin file is not few kilobytes, but few megabytes copying the file might be tricky as the string produced from cat command would be really long, so make sure the SSH client you're using is configured to have a large buffer to scroll up enough and be able to select the whole encoded string until the end of the cat command and copy it to Copy / Paste buffer.

 

5. On Server Host B paste the bas64 encoded binary inside a newly created file

Open with a text editor vim / mc or whatever is available

[root@serverB ~]:# vi inputbinfile-to-encode

Some very paranoid Linux / UNIX systems might not have even a normal text editor like 'vi' if you happen to need to copy files on such one a useful thing is to use a simple cat on the remote side to open a new File Descriptor buffer, like this:

[root@server2 ~]:# cat >> inputbinfile-to-encode <<'EOF'
Paste the string here

 

6. Decode the encoded binary with base64 cmd again

[root@serverB ~]:# base64 –decode outputbin-file.base64 > inputbinfile-to-encode

 

7. Set proper file permissions (the same as on Host A)

[root@serverB ~]:#  chmod +x inputbinfile-to-encode

 

8. Check again the binary file checksum on Host B is identical as on Host A

[root@serverB ~]:# md5sum -b inputbinfile-to-encode
66c4d7b03ed6df9df5305ae535e40b7d *inputbinfile-to-encode

As you can md5sum match on both sides so file should be OK.

 

9. Encoding and decoding files with uuencode


If you are lucky and you have uuencode installed (sharutils) package is present on remote machine to encode lets say an archived set of binary files in .tar.gz format do:

Prepare the archive of all the files you want to copy with tar on Host A:

[root@Machine1 ~]:#  tar -czvf /bin/whatever /usr/local/bin/htop /usr/local/bin/samhain /etc/hosts archived-binaries-and-configs.tar.gz

[root@Machine1 ~]:# uuencode archived-binaries-and-configs.tar.gz archived-binaries-and-configs.uu

Cat / Copy / paste the encoded content as usual to a file on Host B:

Then on Machine 2 decode:

[root@Machine2 ~]:# uuencode -c < archived-binaries-and-configs.tar.gz.uu

 

Conclusion


In this short method I've shown you a hack that is used often by script kiddies to copy over files between pwn3d machines, a method which however is very precious and useful for sysadmins like me who has to admin a paranoid secured servers that are placed in a very hard to access environments.

With the same method you can encode or decode not only binary file but also any standard input/output file content. base64 encoding is quite useful stuff to use also in bash scripts or perl where you want to have the script copy file in a plain text format . Datas are encoded and decoded to make the data transmission and storing process easier. You have to keep in mind always that Encoding and Decoding are not similar to encryption and decryption as encr. deprytion gives a special security layers to the encoded that. Encoded data can be easily revealed by decoding, so if you need to copy between the servers very sensitive data like SSL certificates Private RSA / DSA key, this command line utility tool better to be not used for sesitive data copying.

 

 

Convert PDF .pdf to Plain Text .txt files on GNU / Linux and FreeBSD / pdftotext

Friday, November 16th, 2012

Convert PDF .pdf to .txt Plain Text on GNU / Linux Redhat, Debian, CentOS, Fedora and FreeBSD with pdftotext poppler-utils

If you need to convert Adobe PDF to Plain Text on Linux or FreeBSD, you will have to take a look at a poppler-utils – (PDF Utilities).

For those who wonder why you need at all a .PDF in .TXT, I can think of at least 4 good reasons. 
 

PDF to text convertion on Linux and other UNIX-es is possible through a set of tools called poppler-utils

poppler-utils is installable on most Linux distributions on Debian Ubuntu based Linux-es it is installable with the usual:

noah:~# apt-get install --yes poppler-utils
....

On Fedora it is available and installable from default repositories with yum

[root@fedora~]# yum -y install poppler-utils 

On Mandriva Linux:
[root@mandriva~] # urpmi poppler
....

On FreeBSD (and possibly other BSDs) you can install via ports or install it from binary with:

freebsd# pkg_add -vr poppler-utils
....

Here is a list of poppler-utils contents from the .deb Debian package, on other distros and BSD the /bin content tools are same.
noah:~ # dpkg -L poppler-utils|grep -i /usr/bin/
/usr/bin/pdftohtml
/usr/bin/pdfinfo
/usr/bin/pdfimages
/usr/bin/pdftops
/usr/bin/pdftoabw
/usr/bin/pdftoppm
/usr/bin/pdffonts
/usr/bin/pdftotext

1. Converting  .pdf to .txt 

Converting whole PDF document to TXT is done with:

$ pdftotext PeopleWare-Productive_Projects.pdf PeopleWare-Productive_Projects.txt
 
2. Extracting from PDF to Text file only selected pages

 Dumping to .TXT only specific pages from a PDF file: is done through -f and -l arguments (First and Last) pages number.

$ pdftotext -f 3 -l 10 PeopleWare-Productive_Projects.pdf PeopleWare-Productive_Projects.txt

3. Converting PDF to TXT  protected with password

  $ pdftotext -opw 'Password' Password-protected-file.pdf Unprotected-file-dump.txt

the -opw arguments stand for 'Owner Password'. As suggested by man page -opw will bypass all PDF security restrictions. In PDFs there are file permission password protection as well as user password. 

To remove permissions password protection of file

$ pdftotext -upw 'Password' Password-protected-file.pdf Unprotected-file-dump.txt

 
4. Converting .pdf to .txt and setting type of end of file

Depending on the type of Operating System the TEXT file will be red further, you can set the type of end of lines (for those who don't know it here is the 3 major OSes UNIX, Windows, and MAC end of line codes:

DOS & Windows: \r\n 0D0A (hex), 13,10 (decimal)
Unix & Mac OS X: \n, 0A, 10
Macintosh (OS 9): \r, 0D, 13

$ pdftotext -eol unix PeopleWare-Productive_Projects.pdf
PeopleWare-Productive_Projects.txt

The -eol accepts (mac, unix or dos) as options

A bit off topic but very useful thing is to then listen to converted .txt files using festival.

5. Reading .PDF in Linux Text Console and Terminals

$ pdftotext PDF_file_to_Read.pdf -

Btw it is interesting to mention Midnight Commander ( mcview ), component which supports opening .pdf files in console uses pdftotext for extracting PDFs and visualizing in plain text in exactly same way

Well that's it happy convertion.

Convert doc files to plain text (txt) in terminal / console (tty) on GNU / Linux

Sunday, September 20th, 2009

I was looking for a way to convert Microsoft .doc files to plain text (txt) in Linux directly through terminal.
After some lookup in Google Groups I found ANTIWORD! .
Luckily Debian comes even with a package containing the nice nifty program.
Here is the description of antiword – Converts MS Word files to text, PS and PDF
Fun, desciprtion Eh? 🙂 Ain’t it?
There are some other ways to Convert doc files to plain text, for instance you could use the command catdoc , for example to convert simple .doc to .txt file usecatdoc -a whitepaper.doc.
Another way to convert .doc files to .txt mostly used by developers is via the wvware (nothing to do with vmware!:)) utility.
wvware could directly convert it to html. For example:
wvWare file.doc >file.htmlor
wvText file.doc file.txt
.A lot of things I’ll skip here are well explained in the article Viewing Word files at the command line .
END—–