Posts Tagged ‘Creating’

Linux: logrotate fix log file permissions on newly created logs after rotation

Monday, July 5th, 2021

fix logrotate permission issues of newly logrotated files, howto chown chmod logrotate linux logo

If you have to administer a bunch of Web or Application servers you will definetely end up with some machines that has some logrotate misconfiguration.

Perhaps the most common one sysadmin faces is when you have rotated webserver, proxy, mail server logs that gets gzipped with a date timestamp of the rotation and a brand new files is created by logrotate. Such a thing could be seen on various Linux distributions and even a more corporate prodcution ready Linux – es like CentOS and Fedora occasionally end up with issues caused by improperly created user / group permissions (usually root:root) of logrotate. 

The wrong permissions of usually normally logging to file by a service, happens when the log file will get filled (or matches some thresholds) configured by logrotate respective config, the log rotate mechanism will rename this file gzip / bzip it depending on how it is prepared to behave and opens a new one, however the newly produced log file will not have the  read write  permission which are necessery for the respective service because the service is not running as administrator (root), lets say there is a haproxy daemon running with user / group haproxy, haproxy, like it happeed today on one of our legacy CentOS 6.5 servers.

The sad result is /var/log/haproxy.log or whatever log file stays empty forever even though the service is normally working and you end up blind not seeing what's going on …

To solve the empty file due to logrotate dumping the original file permissions to a wrong one due to misconfiguration or a lack of special configuration it is as easy as setting up the logrotated file to write down the new rotated file to a specic user, this is done with a one line addition of code with a syntax like:

create mode owner group

Below is extract from logrotate man page (man logrotate)

Immediately after rotation (before the postrotate script is run) the log file is created (with the same name as the log file just rotated).  mode  specifies the mode for the log file in octal (the same as chmod(2)), owner specifies the user name who will own the log file, and group specifies the group the log file will belong to. Any of the log file attributes may be omitted, in which case those attributes for the new file will use the same values as the original log file for the omitted attributes. This option can be disabled using the nocreate option.

 Lets say you have following /etc/logrotate.d/haproxy configuration that is instructing logrotate to do the rotation and this will create empty file with root:root after rotate:

root@haproxy2:/etc/logrotate.d# cat haproxy

/var/log/haproxy.log {
    daily
    rotate 52
    missingok
    notifempty
    compress
    delaycompress
    postrotate
        /usr/lib/rsyslog/rsyslog-rotate
    endscript
}

To make /var/log/haproxy.log be owned by haproxy user and group and chmod to certain owner permissions hence, do add inside the block something like: 

 

/var/log/haproxy.log {
….
        create 664 user group
….
}


i.e. :

/var/log/haproxy.log {
….
        create 644 haproxy hapoxy
….
}

To test the configuration do a logrotate config dry run do:

root@haproxy2:/etc/logrotate.d# logrotate -v -d -f /etc/logrotate.d/haproxy
WARNING: logrotate in debug mode does nothing except printing debug messages!  Consider using verbose mode (-v) instead if this is not what you want.

reading config file /etc/logrotate.d/haproxy
Reading state from file: /var/lib/logrotate/status
Allocating hash table for state file, size 64 entries
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
 

Handling 1 logs

rotating pattern: /var/log/haproxy.log  forced from command line (52 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/haproxy.log
  Now: 2021-07-05 21:51
  Last rotated at 2021-07-05 00:00
  log needs rotating
rotating log /var/log/haproxy.log, log->rotateCount is 52
dateext suffix '-20210705'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
compressing log with: /bin/gzip

renaming /var/log/haproxy.log.8.gz to /var/log/haproxy.log.9.gz (rotatecount 52, logstart 1, i 8),
renaming /var/log/haproxy.log.7.gz to /var/log/haproxy.log.8.gz (rotatecount 52, logstart 1, i 7),
renaming /var/log/haproxy.log.6.gz to /var/log/haproxy.log.7.gz (rotatecount 52, logstart 1, i 6),
renaming /var/log/haproxy.log.5.gz to /var/log/haproxy.log.6.gz (rotatecount 52, logstart 1, i 5),
renaming /var/log/haproxy.log.4.gz to /var/log/haproxy.log.5.gz (rotatecount 52, logstart 1, i 4),
renaming /var/log/haproxy.log.3.gz to /var/log/haproxy.log.4.gz (rotatecount 52, logstart 1, i 3),
renaming /var/log/haproxy.log.2.gz to /var/log/haproxy.log.3.gz (rotatecount 52, logstart 1, i 2),
renaming /var/log/haproxy.log.1.gz to /var/log/haproxy.log.2.gz (rotatecount 52, logstart 1, i 1),
renaming /var/log/haproxy.log.0.gz to /var/log/haproxy.log.1.gz (rotatecount 52, logstart 1, i 0),
log /var/log/haproxy.log.53.gz doesn't exist — won't try to dispose of it
renaming /var/log/haproxy.log to /var/log/haproxy.log.1
creating new /var/log/haproxy.log mode = 0644 uid = 106 gid = 112
running postrotate script
running script with arg /var/log/haproxy.log: "
        /usr/lib/rsyslog/rsyslog-rotate
"
 

 

root@haproxy2:/etc/logrotate.d# grep -Ei '106|112' /etc/passwd
haproxy:x:106:112::/var/lib/haproxy:/usr/sbin/nologin

You do it for any other service respectively by editting whatever /etc/logrotate.d/file, lets say postfix's /var/log/maillog should be owned with 644 by postfix:postfix.
 

# cat /etc/logrotate/postfix
/var/log/maillog {
….
        create 664 postfix postfix
….
}

Tags: , , , , , , , , ,
Posted in Linux, System Administration | No Comments »

Build and install Docker Squid Open Proxy image on Kubernetes cluster

Tuesday, October 30th, 2018

docker_container-virtualization-install-open-squid-cache-logo

As Docker containerization is starting to become a standard for installing a brand new servers especially servers who live in Self-Made Clusters based on Orchestration technologites like Kubernetes (k8s) (check out – http://kubernetes.io),

Recently, I've had the task to set-up a Squid Cache (Open Proxy) server on a custom Port number to make it harder for internet open proxy scanners to identify and ship it to a customer.

What is Squid Open Proxy?

An open proxy is a proxy server that is accessible by any Internet user, in other words anyone could access the proxy without any authentication.

Squid is a caching proxy for the Web supporting HTTP, HTTPS, FTP and other protocols.
It reduces bandwidth and improves response times by caching and reusing frequently-requested web pages.
Squid has extensive access controls and makes a great server accelerator.
It runs on most available operating systems, including Windows and is licensed under the GNU GPL.
 

What is Docker?

For those who hear about Docker for a first time, Docker is an open-source software platform to create, deploy and manage virtualized application containers on a common OS such as GNU / Linux or Windows, it has a surrounding ecosystem of tools. Besides its open source version there is also a commercial version of the product by Docker Inc. the original company that developed docker and is today in active help of the project.

docker-engine-components-explained-flow
Docker components  – picture source docker.com

What is Kubernetes?
 

Kubernetes, in short, is an open source system for managing clusters of containers. To do this, it provides tools for deploying applications, scaling those application as needed, managing changes to existing containerized applications, and helps you optimize the use of the underlying hardware beneath your containers.
Kubernetes is designed to be extensible and fault-tolerant by allowing application components to restart and move across systems as needed.
 

Kubernetes is itself not a Platform as a Service (PaaS) tool, but it serves as more of a basic framework, allowing users to choose the types of application frameworks, languages, monitoring and logging tools, and other tools of their choice. In this way, Kubernetes can be used as the basis for a complete PaaS to run on top of; this is the architecture chosen by the OpenShift Origin open source project in its latest release.

Kubernetes-cloud-app-architecture
Kubernetes architecture (shortly explained) – picture source Wikipedia

The Kubernetes project is written in the Google developed Go programming language, and you can browse its source code on GitHub.


Hence, In this article I'll give a brief introuduction on what is Docker and show you, how to easily:

a. Build Docker Image with Ubuntu, Update the system and Install Squid inside the container
using a sample Dockerfile build file

b. Run Docker Image to test deployed Ubuntu Linux and Squid on top of it works fine

c. Push Docker Image to DockerHub (Docker Images Central Official repository)

d. Deploy (Pull and Run) the new built Docker Ubuntu / Squid Open Proxy Image to the Kubernetes Cluster slave nodes – the K8S Cluster was  created using Rancher Enterprise Kubernetes Platform (check out Rancher.net – a bleeding edge tool for k8s quick GUI cluster creation / integration)
 

1. Install Docker Containerization Software Community Edition


Docker containers are similar to virtual machines, except they run as normal processes (containers), that does not use a Hypervisor of Type 1 or Type 2 and consume less resources than VMs and are easier to manage, nomatter what the OS environment is.

Docker uses cgroups and namespace to allow independent containers to run within a single Linux instance.

docker-architecture-explained
Docker Architecture – Picture source docker.com

Below docker install instructions are for Debian / Ubuntu Linux, the instructions for RPM package distros Fedora / CentOS / RHEL are very similar except yum or dnf tool is to be used.

a) Uninstall older versions of docker , docker-engine if present
 

 

apt-get -y remove docker docker-engine docker.io

 

! Previously running docker stuff such as Volumes, Images and networks will be preserved in /var/lib/docker/

b) install prerequired packages and add apt repositories for doko
 

apt-get update
apt-get install -y apt-transport-https ca-certificates wget software-properties-common

 

wget https://download.docker.com/linux/debian/gpg
apt-key add gpg
rm -f gpg

 

Create docker.list apt sources file

 

echo "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | sudo tee -a /etc/apt/sources.list.d/docker.list

 

apt-get update

 

c) check out the docker policy (will list you a multiple installable versoins of docker).

 

apt-cache policy docker-ce

docker-ce:
  Installed: (none)
  Candidate: 17.06.0~ce-0~debian
  Version table:
     17.06.0~ce-0~debian 500
        500 https://download.docker.com/linux/debian stretch/stable amd64 Packages
     17.03.2~ce-0~debian-stretch 500
        500 https://download.docker.com/linux/debian stretch/stable amd64 Packages
     17.03.1~ce-0~debian-stretch 500
        500 https://download.docker.com/linux/debian stretch/stable amd64 Packages
     17.03.0~ce-0~debian-stretch 500

 

d) Install and run docker

 

apt-get -y install docker-ce
systemctl start docker

systemctl-docker-containerization-status-gnu-linux-screenshot

 

systemctl status docker

 

Previously running docker stuff such as Volumes, Images and networks will be preserved in /var/lib/docker/
 

2. Build Docker image with Ubuntu Linux OS and Squid inside


To build a docker image all you need to do is have the Dockerfile (which is docker definitions build file), an Official image of Ubuntu Linux OS (that is provided / downloaded from dockerhub repo) and a bunch of docker commands to use apt / apt-get to install the Squid Proxy inside the Docker Virtual Machine Container

In dockerfile it is common to define for use an entrypoint.sh  which is file with shell script commands definitions, that gets executed immediately after Docker fetches the OS from its remote repository on top of the newly run OS. It is pretty much like you have configured your own Linux distribution like using Linux from Scratch! to run on a bare-metal (hardware) server and part of the installation OS process you have made the Linux to run a number of scripts or commands during install not part of its regular installation process.

a) Go to https://hub.docker.com/ and create an account for free

The docker account is necessery in order to push the built docker image later on.
Creating the account creates just few minutes time.

Build-Docker-Images-with-DockerFile-diagram

b) Create a Dockerfile with definitions for Squid Open Proxy setup

I'll not get into details on the syntax that Dockerfile accepts, as this is well documented on Docker Enterprise Platform official website but in general gettings the basics and starting it is up to a 30 minutes to maximum 1h time.

 

After playing a bit to achieve the task to have my Linux distribution OS (Ubuntu Xenial) with Squid on installed on top of it with the right configuration of SQUID Cacher to serve as Open Proxy I've ended up with the following Dockerfile.

 

FROM ubuntu:xenial
LABEL maintainer="hipo@www.pc-freak.net"

ENV SQUID_VERSION=3.5.12-1ubuntu7 \
    SQUID_CACHE_DIR=/var/spool/squid \
    SQUID_LOG_DIR=/var/log/squid \
    SQUID_USER=proxy

RUN apt-get update \
 && apt-get upgrade && apt-get dist-upgrade && DEBIAN_FRONTEND=noninteractive apt-get install -y squid=${SQUID_VERSION}* \
 && rm -rf /var/lib/apt/lists/*

COPY entrypoint.sh /sbin/entrypoint.sh
COPY squid.conf /etc/squid/squid.conf
RUN chmod 755 /sbin/entrypoint.sh

EXPOSE 3128/tcp
ENTRYPOINT [“/sbin/entrypoint.sh”]

You can download the Dockerfile here.

c) Create entrypoint.sh and squid.conf files

Apart from that I've used the following entrypoint.sh (which creates and sets necessery caching and logging directories and launches script on container set-up) permissions for SQUID proxy  file that is loaded from the Dockerfile on docker image build time.
To have the right SQUID configuration shipped up into the newly built docker container, it is necessery to prepare a template configuration file – which is pretty much a standard squid.conf file with the following SQUID Proxy configuration for Open Proxy

 

acl SSL_ports port 443
acl CONNECT method CONNECT

 

http_access deny !Safe_ports

http_access deny CONNECT !SSL_ports

http_access allow localhost manager
http_access deny manager

 

http_access allow localhost

http_access allow all

http_port 3128


coredump_dir /var/spool/squid

refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern (Release|Packages(.gz)*)$      0       20%     2880
refresh_pattern .               0       20%     4320

 

Once, I've created, the proper Dockerfile configuration, I've made a tiny shell script, that can create / re-create my docker image multiple times. 
Here is the build-docker-image.sh :
 

#!/bin/sh
username='hipod';
password='';
docker login –username="$username" –password=""
DOCKER_ACC=hipod
#DOCKER_REPO=squid-3.5.12-1ubuntu7
DOCKER_REPO=squid-ubuntu
IMG_TAG=latest

docker build -t $DOCKER_ACC/$DOCKER_REPO:$IMG_TAG .
docker push $DOCKER_ACC/$DOCKER_REPO:$IMG_TAG


– You can download build-docker-image.sh script here

The script uses the docker login command to authenticate non-interactively to https://hub.docker.com
docker build command with properly set DOCKER_ACC (docker account – which is the username of your hub.docker.com account as I've pointed earlier in article), then DOCKER_REPO (docker repository name) – you can get it either from a browser, after you've logged in to dockerhub or assuming you know your username, it should look like:
https://hub.docker.com/u/your-username-name – for example mine is hipod with repository name squid-ubuntu, my squid-ubuntu docker image build is here, you'll also need to provide the password inside the script or if you consider it a security concern, instead type manually from command line docker login and authenticate in advance before running the script, finally the last line docker push pushes to remote docker hub the new build of Ubuntu + SQUID Proxy with a predefined TAG that in my case is latest (as this is my latest build of Squid – if you need a multiple version number of Squid repository just change the tag to the version tag line number.

 

d) Use the script to build Squid docker image

build-docker-image-terminal-screenshot-linux

Next run the script to make and push into docker your new image:
 

sh build-docker-image.sh

 

Please consider that in order to work with docker hub push / pull, you will need to have a firewall that allows connection to dockerhub site repo, if for some reason the push / pull fails, check closely your firewall as it is the most likely cause for failure.

 

3. Run the new docker image to test Squid runs as expected

To make sure the docker image runs properly, you can test it on any machine that has docker.io installed, this is done with a simple cmd:
 

docker run -d –restart=always -p 3128:3128 hipod/squid-ubuntu:latest

 


The -d option tells docker to background process / run in detached mode
 -p option tells docker to expose port (e.g. make NAT with iptables from the docker virtual container with Linux OS + SQUID listening inside the container on port 3128 to the TCP / IP 3128 server port).
You can use iptables to check the created Network Address Translation rules.

–restart=always option sets the docker restart policy (e.g. when the container is terminating it tells the container to restart the container (OS) after exit), there, you can use as a resetart policy (no, on-failure[:max_retries] , unless-stopped)

For clarity I've created one liner script run-squid.sh you can download here.
 

sh run-squid.sh

You can use a
 

ps aux|grep -i docker
 

root     13169  0.2  0.4 591300 77984 ?        Ssl  13:50   0:09 /usr/bin/dockerd -H fd://
root     13176  0.0  0.2 408244 34972 ?        Ssl  13:50   0:03 docker-containerd –config /var/run/docker/containerd/containerd.toml
root     20875  0.0  0.0 132348   948 pts/1    S+   14:52   0:00 grep -i docker


to see it running as well as the:

 

docker ps -a
 

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                      PORTS               NAMES
d2eb7ab635cf        c5b0f61227cd        "/bin/sh -c 'apt-get…"   12 minutes ago      Exited (1) 3 minutes ago                        trusting_elion
18476f546562        c5b0f61227cd        "/bin/sh -c 'apt-get…"   37 minutes ago      Exited (1) 37 minutes ago                       admiring_wilson


To connect to the running container later you can use docker attach ID_of_container

 

docker attach d2eb7ab635cf

 

 


command to see the new container runs as well as attach to the newly spawned container

 

4. Deploying Dockerized SQUID Open Proxy Cache server to Kubernetes cluster

 

My task was to deploy the newly built squid doko image to remote K8s cluster which was set as a default cluster via a context in  .kube/config/  or manually set via:

 

kubectl config use-context my.k8s-cluster.net


I've used the following YAML file with kubectl to deploy:

 

—
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: squid-app-33128
  name: squid-app-33128
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: squid-app-33128
    spec:
      containers:
        – name: squid
          image: hipod/squid-ubuntu:latest
          imagePullPolicy: Always
          ports:
          – containerPort: 3128
            protocol: TCP
          volumeMounts:
            – mountPath: /var/spool/squid
              name: squid-cache
            – mountPath: /var/log/squid
              name: squid-log
          livenessProbe:
            tcpSocket:
              port: 3128
            initialDelaySeconds: 40
            timeoutSeconds: 4
      volumes:
        – name: squid-cache
          emptyDir: {}
        – name: squid-log
          emptyDir: {}


The task included to deploy two different Open Proxy squid servers on separate ports in order to add them external cluster Ingress load balancing via Amazon AWS, thus I actually used following 2 yaml files.

1. squid-app-33128.yaml
2. squid-app-33129.yaml

The actual deployment command was like that:

 

# deploys pods and adds services to kubernetes cluster

kubectl create -f squid-app-33129.yaml -f squid-app-33128.yaml -f squid-service-33129.yaml squid-service-33128.yaml

 

You can download the 2 squid-service .yaml below:

1. squid-service-33129.yaml
2. squid-service-33128.yaml

The service is externally exposed via later configured LoadBalancer to make the 2 squid servers deployed into k8s cluster accessible from the Internet by anyoneone without authorization (as a normal open proxies) via TCP/IP ports 33128 and 33129.
 

Conclusion


Below I explained a few easy steps to follow to;


– build docker image Ubuntu + Squid
– test the image
– deploy the image into a previously prepared k8s cluster

Though it all looks quite simplistic I should say creating the .yaml file took me long. Creating system configuration is not as simple as using the good old .conf files and getting used with the identation takes time.

Now once the LB are configured to play with k8s, you can enjoy the 2 proxy servers. If you need to do some similar task and you don't have to do it for a small fee, contact me.

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,
Posted in Cloud services, Docker, Educational | No Comments »

Creating multi-part zip archives in Linux with 7zip command to transfer large zip files data in parts

Monday, December 1st, 2014

creating-multi-part-zip-archives-in-Linux-debian-ubuntu-fedora-centos-rhel-with-7z-command-to-transfer-large-files-data-in-parts
Recently, I've blogged on how to move large files from source to destination server in parts on a slow / restricted networks or whenever the media is limtied in size. This is not a common scenario but it happens so if you're admin sooner or later you will need that. I give example with UNIX's split and unrar. However strip's file naming can get you insane (in case if you don't want to use cstrip command – split a file into sections determined by context lines instead) plus normal split Linux / *nix command doesn't support compression and encryption. On the other side on many Company internal Networks with Windows server hosts running – Winblows (2003, 2008, 1012) for security purposes it might be that WinRar is not installed, thus you might need to transfer the file parted between the GNU / Linux server and Windows server in standard OS supported by Windows ZIP format. Assuming that you have root (admin) access to the Linux host you can then archive your file in parts using ZIP encryption algorithm with 7zip.

1. Installing 7zip on CentOS / Fedora / RHEL and other Redhat based Linuces

If the Linux server is running:
Fedora / CentOS  / RHEL and you don't have 7zip installed yet install it with:

yum -y install p7zip

According to distros version  it might be the name could be a bit different if p7zip is different to find the one you need search with:

yum search p7zip

and install whatever you need
 

2. Installing 7zip on Debian / Ubuntu and other Debian based servers

apt-get install –yes p7zip-full

Depending on Deb based distro just like with fedora if p7zip-full pack is not installable, check 7zip's package distro version:

apt-cache search p7zip

 

3. Archiving ZIP file in multiple (sized) parts on GNU / Linux

7z a -v512m Large-file-separated-in-multi-parts.zip Large-Many-Gigabytes-File.SQL

This would output multiple files:

Large-file-separated-in-multi-parts.zip.001, Large-file-separated-in-multi-parts.zip.002, Large-file-separated-in-multi-parts.zip.003, Large-file-separated-in-multi-parts.004 etc.

If you want to add security to the transferred file to protect newly created ZIP archive with password use following command:

7z a -v512m Large-file-separated-in-multi-parts.zip Large-Many-Gigabytes-File.SQL

 

7-Zip [64] 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=bg_BG.UTF-8,Utf16=on,HugeFiles=on,2 CPUs)
Scanning

Creating archive Large-file-separated-in-multi-parts.zip


Enter password (will not be echoed) :
…

Once you have transferred all the many parts via (SSH/ FTPS or not preferrably HTTP / HTTPS / FTP) place them in the same folder and use Windows standard ZIP to unarchive.

If the archived 7zip files are to be unarchived on another Linux host (in case if multi part zip transfer is between Linux -> Linux hosts) to unarchive, parted files:

7z x Large-file-separated-in-multi-parts.zip.* …

Tags: , , , , , , , , , , , ,
Posted in Everyday Life, Linux, Linux and FreeBSD Desktop, System Administration, Various, Windows | 1 Comment »

Create video from linux console / terminal – Record ssh terminal session as video with asciinema, showterm, termrecord

Thursday, August 21st, 2014

/var/www/images/asciinema-create-and-upload-ascii-terminal-console-videos-debian-gnu-linux-screenshot
You probably already know of existence of two Linux commands available by default across all Linux distributions script – which makes a text based save of all commands executed on console and scriptreplay – which playbacks saved script command typescripts. Using this two you can save terminal sessions without problem, but in order to play them you need to have a Linux / UNIX computer at hand.
However If you want to make a short video record displaying what you have done on Linux console / terminal, you have few other options with which you can share your Linux terminal sessions on the web. In this short article I will go through 3 popular tools to do that – asciinema, showterm and termrecord.

1. Asciinema Current most popular tool to create video from Linux terminal

Here is how ASCIINEMA's website describes it:

"Asciinema is a free and open source solution for recording the terminal sessions and sharing them on the web."

apt-get –yes install python-pip

To install it with pip – python package installer

pip install asciinema

Or if the machine is in DMZ secured zone and have access to the internet over a Proxy:

pip install –proxy=http://internet-proxy-host.com:8080 asciinema

It will get installed in /usr/local/bin/asciinema to make a terminal screen video capture just launch it (nomatter if it is privileged or non-privileged user):

asciinema

To finalize and upload the recorded terminal session, just type exit (to exit the shell), hopefully it will get you an upload link.

exit

You can claim authorship on video you issue:

asciinema auth

Use can then embed the new Linux terminal session video to your website.
 

2. ShowTerm – "It's showtime in a terminal near you!"

ShowTerm have same features as AsciiNema. Just like AsciiNema, what it does is it creates a record of your terminal session and then uploads it to showterm.io website, providing you a link over which you can share your terminal lesson / ascii art video / whatever with your friends. ShowTerm is written in, the world famous Ruby on Rails – ruby web development framework, so you will need to have ruby programming language installed before use. As showterm uses the Internet to upload video, so it is not really an option to create videos from remote terminal session on servers which are in DMZ with no access to the internet, I will explain in a little while how to create video of your terminal / console for private purpose on local server and then share it online on your own site.

a) To install ShowTerm:

– First be sure to have ruby installed:

On Debian / Ubuntu and derives deb Linux, as supersuser:

apt-get install –yes ruby curl

On CentOS / RHEL / Fedora Linux

yum -y install ruby curl

NB! curl is real requirement but as showterm.io website recommends downloading the script with it and later same curl tool is used to upload the created showterm file to http://showterm.io .

– Then to finalize install, download showterm script and make it executable

curl showterm.io/showterm > ~/bin/showterm

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                       Dload  Upload   Total   Spent    Left  Speed
100  2007  100  2007    0     0   2908      0 –:–:– –:–:– –:–:–  8468
mkdir ~/bin
chmod +x ~/bin/showterm

This will save the script into your home folder ~/bin/showterm

b) Using showterm

To run it to create video from your terminal simply start it and do whatver you will in terminal.

~/bin/showterm

After you're done with the video you like type exit

exit

create-video-from-your-linux-console-terminal-with-showterm-screenshot

Note that if your server is behind a proxy curl will not understand proxy set inside Linux shell variable with http_proxy var, to upload the file if you're behind a proxy you will have to pass to curl –proxy setting, once you get the curl line invoked after failure to upload use something like:

curl –proxy $(echo $http_proxy)  https://showterm.herokuapp.com/scripts –data-urlencode cols=80 –data-urlencode lines=24 –data-urlencode scriptfile@/tmp/yCudk.script –data-urlencode timingfile@/tmp/lkiXP.timing

Where assuming proxy is defined already inside http_proxy shell variable.

 

3. Creating video from your terminal / console on Linux for local (private) use with TermRecord

In my humble view TermRecord is the most awesome of all the 3, as it allows you to make records with an own generated Javascript based video player and allows you to keep the videos on your own side, guaranteeing you independence of external services. Its
 

pip install TermRecord

TermRecord -o /tmp/session.html

 

You can further access the video in a local browser in Firefox / Chrome / Epiphany type in URL address bar:

/tmp/session.html to play the video

create-video-from-terminal-console-on-gnu-linux-howto-screenshot-with-termrecord

TermRecord uses term.js javascript to create the video web player and play the video which is directly encoded inside session.html.
If you want to share the video online, place it on your webserver and you're done 🙂
Check out my TermRecord generated video terminal sample session here.
 

Tags: , , , , , , , , , , , , , , , ,
Posted in Entertainment, Everyday Life, Linux, Linux and FreeBSD Desktop, Linux Audio & Video, System Administration, Various | No Comments »

Quick way to access remotely your GNU / Linux Desktop – Access Linux Desktop from Mac and Windows 7

Tuesday, August 5th, 2014

how-to-access-linux-host-from-microsoft-windows-or-mac-client-xrdp-tightvnc-native-way-logo
For M$ Windows users its always handy to have remote access to your home PC or notebook via Remote Desktop (RDP) protocol.

However in GNU / Linux, there is no native implementation of RDP protocol. So if you're using Linux as your Desktop like me you will probably want to be able to access the Linux system remotely not only via terminal with SSH using (Putty) or MobaXTerm all in one tabbed Windows terminal program but also be able to use your Linux GNOME / KDE Graphical environment from anywhere on the Internet.

This will make you ponder – Is it possible to access Linux Desktop via proprietary RDP protocol and if not how you can achieve remote GUI access to Linux?

1. Using Linux Xorg and Xming Xserver for Windows

Most people should already know of Linux ability to start multiple Xserver sessions remotely which is the native way to access between two Linux hosts or access remotely Linux from other Linux UNIX like OS. It is also possible to use xinit / startx / xhost commands to establish remotely connection to new or running Linux (Xorg) Xserver by using them in combination with XMing – XServer for Windows running on the Windows host and Debian package (x11-xserver-utils) – providing xhost cmd, however this method is a bit complicated and not so convenient.

I used to be using this method XMing (whose mirror is here), earlier in my university years to use remotely my Debian Linux from  Windows 98 and this works perfectly fine.

2. Using RDP emulation with XRDP server

in order to be able to access your desk from any friend or computer club in the world using standard available in MS Windows Remote Desktop client (mstsc.exe).
There is also another alternative way by using Windows Desktop sharing RDP experimental server xrdp:
 

apt-cache show xrdp |grep -i descr -A 3
Description: Remote Desktop Protocol (RDP) server
 Based on research work by the rdesktop project, xrdp uses the Remote
 Desktop Protocol to present a graphical login to a remote client.
 xrdp can connect to a VNC server or another RDP server.

To make your Linux host accessible via RDP:

On Debian / Ubuntu etc. deb based Linux:

 

apt-get update
apt-get install xrdp
 
$ /etc/init.d/xrdp status
Checking status of Remote Desktop Protocol server xrdp                                             [ OK ]
Checking status of RDP Session Manager sesman

/etc/init.d/xrdp start

On  Fedora Linux:
 

yum -y install xrdp
systemctl enable xrdp.service
systemctl start xrdp.service
systemctl enable xrdp-sesman.service
systemctl start xrdp-sesman.service


It is possible to access remote Linux host using xrdp RDP server, but this will only work in older releases of mstsc.exe (Windows XP / Vista / 2003) and will not work on Windows 7 / 8, because in MS Windows 7 and onwards RDP proto version has changed and the client no longer has compatability with older mstsc releases. There is a work around for this for anyone who stubbornly want to use RDP protocol to access Linux host. If you want to connect to xrdp from Windows 7 you have to copy the old RDP client (mstsc.exe and mstscax.dll) from a WinXP install to the Windows 7 box and run it independently, from the default installed ones, anyways this method is time consuming and not really worthy …

3. Using the VNC withTightVNC server / client

 

Taking above in consideration, for me personally best way to access Linux host from Windows and Mac is to use simply the good old VNC protocol with TightVNC.

TightVNC is cross-platform free and open source remote Desktop client it uses RFB protocol to control another computer screen remotely.

To use tightvnc to access remote Debian / Ubuntu – deb based Linux screen, tightvncserver package has to be installed:

apt-cache show tightvncserver|grep -i desc -A 7
Description-en: virtual network computing server software
 VNC stands for Virtual Network Computing. It is, in essence, a remote
 display system which allows you to view a computing `desktop' environment
 not only on the machine where it is running, but from anywhere on the
 Internet and from a wide variety of machine architectures.
 .
 This package provides a server to which X clients can connect and the
 server generates a display that can be viewed with a vncviewer.
—
 

apt-get –yes install tightvncserver
…


TightVNCserver package is also available in default repositories of Fedora / CentOS / RHEL and most other RPM based distros, to install there:
 

yum -y install tightvnc-server
…


Once it is installed to make tightvncserver running you have to start it (preferrably with non-root user), usually this is the user with which you're using the system:

tightvncserver

You will require a password to access your desktops.

Password:
Verify:   
Would you like to enter a view-only password (y/n)? n

New 'X' desktop is rublev:4

Creating default startup script /home/hipo/.vnc/xstartup
Starting applications specified in /home/hipo/.vnc/xstartup
Log file is /home/hipo/.vnc/rublev:4.log

 

tightvncserver-running-in-gnome-terminal-debian-gnu-linux-wheezy-screenshot

To access now TightVncserver on the Linux host Download and Install TightVNC Viewer client

note that you need to download TightVNC Java Viewer JAR in ZIP archive – don't install 32 / 64 bit installer for Windows, as this will install and setup TightVNCServer on your Windows – and you probably don't want that (and – yes you will need to have Oracle Java VM installed) …
 

tightvnc-viewer-java-client-running-on-microsoft-windows-7-screenshot

Once unzipped run tightvnc-jviewer.jar and type in the IP address of remote Linux host and screen, where TightVNC is listening, as you can see in prior screenshot my screen is :4, because I run tightvnc to listen for connections in multiple X sessions. once you're connected you will be prompted for password, asker earlier when you run  tightvncserver cmd on Linux host.

If you happen to be on a Windows PC without Java installed or Java use is prohibited you can use TightVNC Viewer Portable Binary (mirrored here)

/images/tightvnc-viewer-portable-windows-7-desktop-screenshot

If you have troubles with connection, on Linux host check the exact port on which TightVncServer is running:
 

ps ax |grep -i Tightvnc

 8630 pts/8    S      0:02 Xtightvnc :4 -desktop X -auth /var/run/gdm3/auth-for-hipo-7dpscj/database -geometry 1024×768 -depth 24 -rfbwait 120000 -rfbauth /home/hipo/.vnc/passwd -rfbport 5904 -fp /usr/share/fonts/X11/misc/,/usr/share/fonts/X11/Type1/,/usr/share/fonts/X11/75dpi/,/usr/share/fonts/X11/100dpi/ -co /etc/X11/rgb

Then to check, whether the machine you're trying to connect from doesn't have firewall rules preventing the connection use (telnet) – if installed on the Windows host:
 

telnet www.pc-ferak.net 5904
Trying 192.168.56.101…
Connected to 192.168.56.101.
Escape character is '^]'.
RFB 003.008

telnet> quit
Connection closed.

remote-connection-via-tightvnc-to-linux-host-from-windows-7-using-tightvnc-java-client-screenshot
 

Tags: , , , , , , , , , , , , , , , ,
Posted in Everyday Life, Linux and FreeBSD Desktop, Remote System Administration, System Administration | 3 Comments »

How to protect Munin Web statistics with password on GNU / Linux

Thursday, October 27th, 2011

I just installed munin to track in web the performance of few Debian servers. I’ve configured munin to open via a Virtualhosts in Apache. As its always wise to protect any statistics data about the server from the unwanted possible security violators, I decided to protect Munin with Apache .htaccess.

The munin htmldir output dir is configured to be in /var/www/munin, hence I protected my munin with password by:

1. Creating .htaccess file in /var/www/munin with following content

AuthUserFile /etc/apache2/.munin_htpasswd
AuthGroupFile /dev/null
AuthName EnterPassword
AuthType Basic

require user admin

2. Creating /etc/apache2/.munin_htpasswd with htpasswd (htaccess password generator cmd)

debian:/var/www/munin# htpasswd -c /etc/apache2/.munin_htpasswd admin
New password:
Re-type new password:
Adding password for user admin

Another important thing I had to do is set my VirtualHost file to be configured with AllowOverride All , if AllowOverride All is missing the .htaccess and .htpasswd are not red at all.
Afterwards munin is protected with password, and when my virtualdomain where munin lays e.g. http://munin.mydomain.com is accessed the .htpasswd password dialog pops up 😉

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , ,
Posted in Linux, System Administration, Web and CMS | No Comments »