Posts Tagged ‘repository’

How to set repository to install binary packages on amd64 FreeBSD 9.1

Friday, January 11th, 2013

Though, it is always good idea to build from source for better performance of Apache + MySQL + PHP, its not worthy the time on installing minor things like; trafshow, tcpdump or deco (MC – midnight commander like native freebsd BSD program).

If you're on a 64 bit version of FreeBSD ( amd64) 9.1 and you try to install a binary package with;

freebsd# pkg_add -vr vim

Ending up with an error;

Error: Unable to get ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9.1-release/Latest/vim.tbz: File unavailable (e.g., file not found, no access)
pkg_add: unable to fetch 'ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9.1-release/Latest/vim.tbz' by URL
pkg_add: 1 package addition(s) failed

The error is caused by lack of special packages-9.1-release directory existing on FreeBSD.org servers. I've realized this after doing a quick manual check opening ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64. The existing URL containing working fbsd 9.1 binaries is:

ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9-current/Latest/
h

You will have to set a repository for FreeBSD 9.1 amd64 packages manually with cmd:
freebsd# echo $SHELL
/bin/csh
freebsd# setenv PACKAGESITE ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9-current/Latest/

If you're on bash shell use export instead:

freebsd# export PACKAGESITE="ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9-current/Latest/"

To make ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9-current/Latest/ as a permanent binary repository:

echo 'setenv PACKAGESITE ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9-current/Latest/' >> /root/.cshrc

or

echo 'export PACKAGESITE="ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9-current/Latest/"' >> /root/.bashrc

Now, pkg_add as much as you like 😉

Finding nearest package software repository in Debian GNU / Linux

Thursday, September 6th, 2012

finding debian Ubuntu package repository icon

I’m about to chage the good old computeres until this very moment this blog and few other website were running on. Right now, I’m installing the brand new machine Lenovo ThinkCentre Edge great and hopefully powerful enough machine to take care for the periodic occuring high traffic loads which break up webserver or SQL server. Well anyways, I just installed latest Debian GNU / Linux on this brand new piece of iron. During install I couldn’t connect the PC to network so Debian install was unable to determine, the nearest Debian package repository, hence after completing install and anually configuring Debian network . Because during install the system had no connection with the Internet, no proper package repository definitions were present in /etc/apt/sources.list, hence I had to find the nearest package software repository. Normally one can check in Debian official WorldWide Mirror sites full address list and determine by some rationalization with ping or / and a manual package download which repo is quickest. There is thanksfully a better automated way one can determine the closest deb Debian / Ubuntu located repository with netselect-apt.

Here is apt-cache description:

debian:~# apt-cache search netselect-apt
netselect-apt - speed tester for choosing a fast Debian mirror

Using the tool is trivial, just install, run it and it does all 4 u 🙂

1. Install netselect-apt

debian:~# apt-get install --yes netselect-apt

2. Run it

debian:~# netselect-apt
....
130/debian/); keeping only under first name.
netselect: 2 (2 active) nameserver request(s)...

Duplicate address 200.236.31.3 (http://debian.c3sl.ufpr.br/debian/, http://ftp.br.debian.org/debian/); keeping only under first name.
netselect: 1 (1 active) nameserver request(s)...
Running netselect to choose 1 out of 383 addresses.
.............................................
The fastest server seems to be:
http://debian.telecoms.bg/debian/

Writing sources.list.
sources.list exists, moving to sources.list.1346964774
Done.

As you can see from output, the tool finds the quickest download deb repository and generate /etc/apt/sources.list file in current directory, where it is run in, in this exact case it creates it in root user home dir – e.g. in /root/ directory. Once the repo address is found you can copy paste it with some text editor to /etc/apt/sources.list or move it over /etc/apt/sources.list;

debian:~# cp -rpf /etc/apt/sources.list /etc/apt/sources.list.$(date +%d_%m_%Y|sed -e 's/^ *//')
debian:~# mv /root/sources.list /etc/apt/sources.list

Just in case as I always make first copy of original sources.list, this is not necessery but IMHO a generally good sysadmin habit 🙂

Besides netselect-apt, which automatically choose between all available list of software repo servers, there is also netselect tool. netselect does basically the same the only difference is one has to manually pass by as arguments deb package repositories and the tool then does tests and returns which is the overall quickest deb download source.

netselect is definitely useful if you have started few own mirror of repositories and want to determine which is the best among them.

Here is how netselect is used:

# netselect -vv ftp.fceia.unr.edu.ar ftp.kulnet.kuleuven.ac.be \
ftp.cdrom.com ftp.debian.org ftp.de.debian.org
ftp.fceia.unr.edu.ar 2792 ms 23 hops 100% ok ( 1/ 1) [ 9213]
ftp.kulnet.kuleuven.ac.be 9999 ms 30 hops 0% ok
ftp.cdrom.com 94 ms 8 hops 100% ok (10/10) [ 169]
ftp.debian.org 46 ms 15 hops 100% ok (10/10) [ 115]
ftp.de.debian.org 9999 ms 30 hops 0% ok

According to above output, the “best reachable” (quickest) repository is the one to which are the least miliseconds – ftp.debian.org

Thanks to- https://www.wzdftpd.net/blog/index.php? -for the nice Debian package box picture – all copyrights belong to respective authors and licensing.
Cheers ! 🙂

Installing XMMS on Debian Squeeze from a Package / Installing XMMS on Debian – the debian way

Tuesday, July 17th, 2012

installing xmms on debian squeeze linux playing free software song green skin screenshot

I use Debian Linux for my desktop for quite some time; Even though there are plenty of MP3 / CD players around in Debian, I’m used to the good old XMMS, hence I often prefer to use XMMS to play my music instead of newer players like RhythmBox or audacious.
Actually audacious is not bad substitute for XMMS and is by default part of Debian but to me it seems more buggy and tends to crash during playing some music formats more than xmms ….

As most people might know, XMMS is no longer supported in almost all modern Linux distributions, so anyone using Debian, Ubuntu or other deb derivative Linux would have to normally compile it from source.
Compiling from source is time consuming and I think often it doesn’t pay back the effort. Thanksfully, though not officially supported by Debian crew XMMS still can be installed using a deb xmms prebuilt package repository kindly provided by a hacker fellow knuta.

Using the pre-build deb packages, installing xmms on new Debian installs comes to:

debian:~# echo 'deb http://www.pvv.ntnu.no/~knuta/xmms/squeeze ./' >> /etc/apt/sources.list
debian:~# echo 'deb-src http://www.pvv.ntnu.no/~knuta/xmms/squeeze ./' >> /etc/apt/sources.list
debian:~# apt-get update && apt-get -y install xmms

There are also deb xmms built for Ubuntu, so Ubuntu users could install xmms using repositories:

deb http://www.pvv.ntnu.no/~knuta/xmms/karmic ./
deb-src http://www.pvv.ntnu.no/~knuta/xmms/karmic ./
That’s all now xmms is ready to use. Enjoy 🙂

How to add repository manually from command line in Ubuntu Linux

Sunday, January 8th, 2012

I'm on a way trying to install Free Mega Games Pack and I'm facing troubles in following the instructions to add a the latest development wine version described on http://www.winehq.org/download/ubuntu
The guys from WineHQ has to update the wine install instructions, since the instructions are targetting older versions of Ubuntu which are not compatible with newer Ubuntus which comes natively with Unity
In order to complete the step in adding the WineHQ Ubuntu PPA development repository my only way was to add it using command line.
Here is how:

root@ubuntu:~# apt-add-repository ppa:ubuntu-wine/ppa
You are about to add the following PPA to your system:
Latest official WineHQ releases
Welcome to the Wine Team PPA. Here you can get the latest available Wine betas for every supported version of Ubuntu. This PPA is managed by Scott Ritchie, and is a replacement for the WineHQ budgetdedicated.com repository used for Jaunty and earlier.
More info: https://launchpad.net/~ubuntu-wine/+archive/ppa
Press [ENTER] to continue or ctrl-c to cancel adding it
Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --secret-keyring /tmp/tmp.bvo21sFWKG --trustdb-name /etc/apt/trustdb.gpg --keyring /etc/apt/trusted.gpg --primary-keyring /etc/apt/trusted.gpg --keyserver hkp://keyserver.ubuntu.com:80/ --recv 883E8688397576B6C509DF495A9A06AEF9CB8DB0
gpg: requesting key F9CB8DB0 from hkp server keyserver.ubuntu.com
gpg: key F9CB8DB0: public key "Launchpad PPA for Ubuntu Wine Team" imported
gpg: no ultimately trusted keys found
gpg: Total number processed: 1
gpg: imported: 1 (RSA: 1)

Similarly adding a PPA repository on Debian is also possible by using a little shell script add-apt-repository.sh . add-apt-repository.sh simulates what ubuntu's apt-add-repositry python script does.

It is educative to mention PPA stands for (Personal package Archive) and the difference between normal repository and PPA is mainly in the fact that PPA repositories makes a package distributed by the repository like the native Ubuntu packages issued by Canonical.
Once for example a new version of a file is placed in PPA deb package repository, the newer package will be automatically installed to the system using it.

Easy way to look for irregularities and problems in log files / Facilitate reading log files on GNU / Linux and FreeBSD

Thursday, November 24th, 2011

LogWatch logo picture check Logcheck Linux BSD look for irregularities in log files

As a System Administrator I need to check daily the log files produced on various GNU / Linux distributions or FreeBSD. This can sometimes take too much time if the old fashioned way using the normal system tools cat, less and tail etc. is used.

Reading logs one by one eats too much of my time and often as logs are reviewed in a hurry some crucial system irregularities, failed ssh or POP3 / Imap logins, filling disk spaces etc. are missed.

Therefore I decided to implement automated log parsing programs which will summary and give me an overview (helicopter view) on what were the system activities from the previous day (24h) until the moment I logged the system and issued the log analyzer program.
There are plenty of programs available out there that does “wide scale” log analysis, however there are two applications which on most GNU / Linux and BSD systems had become a de-facto standard programs to scan system log files for interesting lines.

These are:
 

  • 1. logwatchsystem log analyzer and reporter
  • 2. logcheckprogram to scan system log files for interesting lines

1. logwatch is by default installed on most of the Redhat based Linux systems (Fedora, RHEL, CentOS etc.). On Debian distributions and as far as I know (Ubuntu) and the other deb based distros logwatch is not installed by default. Most of the servers I manage these days are running Debian GNU / Linux so, to use logwatch I needed to install it from the available repository package, e.g.:

debian:~# apt-get install logwatch
...

logwatch is written in perl and with some big files to analyze, parsing them might take hell a lot of time. It does use a bunch of configuration scripts which defines how logwatch should read and parse the various services logwatch support by default. These conf scripts are also easily extensible, so if one has to analyze some undefined service in the conf files he can easily come up with a new conf script that will support the service/daemon of choice.Using logwatch is very easy, to get an overview about server system activity invoke the logwatch command:

debian:~# logwatch
################### Logwatch 7.3.6+cvs20080702-debian (07/02/08) ####################
Processing Initiated: Thu Nov 24 05:22:07 2011
Date Range Processed: yesterday
( 2011-Nov-23 )
Period is day.
Detail Level of Output: 0
Type of Output/Format: stdout / text
Logfiles for Host: debian
 ################################################# 

——————— dpkg status changes Begin ————- 

Upgraded:
libfreetype6 2.3.7-2+lenny7 => 2.3.7-2+lenny8
libfreetype6-dev 2.3.7-2+lenny7 => 2.3.7-2+lenny8

———————- dpkg status changes End ————————-

——————— httpd Begin ————————

Requests with error response codes
400 Bad Request
HTTP/1.1: 2 Time(s)
admin/scripts/setup.php: 2 Time(s)
401 Unauthorized


———————- vpopmail End ————————-

——————— Disk Space Begin ————————

Filesystem Size Used Avail Use% Mounted on
/dev/md0 222G 58G 154G 28% /

———————- Disk Space End ————————-

###################### Logwatch End #########################

The execution might take up from 10 to 20 seconds up to 10 or 20 minutes depending on the log files size and the CPU / RAM hardware on the machine where /var/log/… logs will be analyzed.

logwatch output can be easily mailed to a custom mail address using a crontab if the server runs a properly configured SMTP server. Using a cron like:

00 5 * * * /usr/sbin/logwatch | mail -s "$(hostname) log files for $(date)"

Here is time to make a note that logwatch is ported also to FreeBSD and is available from BSD’s port tree, from a port with path:

/usr/ports/security/logcheck

2. logcheck is another handy program, which does very similar job to logwatch . The “interesting” information it returns is a bit less than compared to logwatch

The good thing about logcheck is that by default it is made to mail every 1 hour a brief data summary which might be of an interest to the sys admin.
Logcheck is available for install on RedHat distros via yum and has existing package for Debian as well as a port for FreeBSD under the port location /usr/ports/security/logcheck

To install on logcheck on Debian:

debian:~# apt-get install logcheck
...

After installation I found it wise to change the default mailing time from each and every hour to just once per day to prevent my email from overfilling with “useless” mails.

This is done by editting the default cron tab installed by the package located in /etc/cron.d/logcheck

The default file looks like so:

# /etc/cron.d/logcheck: crontab entries for the logcheck package
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
@reboot logcheck if [ -x /usr/sbin/logcheck ]; then nice -n10 /usr/sbin/logcheck -R; fi
2 * * * * logcheck if [ -x /usr/sbin/logcheck ]; then nice -n10 /usr/sbin/logcheck; fi
# EOF

To change it run only once per day its content should looks something like:

# /etc/cron.d/logcheck: crontab entries for the logcheck package
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
@reboot logcheck if [ -x /usr/sbin/logcheck ]; then nice -n10 /usr/sbin/logcheck -R; fi
2 5 * * * logcheck if [ -x /usr/sbin/logcheck ]; then nice -n10 /usr/sbin/logcheck; fi
# EOF

Altering it that way the log summary interesting info analysis will be sent on mail every day in 05:02 a.m.
Changing the default email logcheck will ship its log analyzer report emails on deb based distros is done via editting the file:

/etc/logcheck/logcheck.conf

And changing the SENDMAILTO=”” variable to point to the appropriate admin email email addr.
 

Graphical representation of Open Source history generated from SVN and GIT commits with Gource – Software version control visualization

Thursday, September 1st, 2011

A friend of mine has sent me a link to awesome videos produced with an open source tool called Gource

The tool is really awesome as its capable of producing amazing graphical visualizations from source tree repositories obtained from software SVN or GIT repositories

Someone used it to create a wonderful videos and relate the graphical visualization with wondeful music to show graphically the Software development for the main Free Software Projects. 😉

The points which move around in videos are graphical representation of the repository source, the nodes which buzzle around are users who commit source in the project repositories.
The whole graphical representation is being generated based on all the latest source hold inside the software repository, on top of the videos its visible the date of each of the source commits.
Below are few of the nice videos, the rest are available for checkout in Youtube, Enjoy! 😉

OpenOffice Development – Graphical representation

PHP Development – Graphical representation

History of Python development since 1990 with Gource

Development of MySQL 2000-2010 – Graphical representation

Perl development history in less then 4 minutes. Visualized with gource.

Installing HTOP on CentOS 5.5 OpenVZ Linux server from source

Friday, July 22nd, 2011

Htop Cool picture logo / htop on CentOS OpenVZ

Lately, I’m basicly using htop‘s nice colourful advanced Linux top command frontend in almost every server I manage, therefore I’ve almost abondoned top usage these days and in that reason I wanted to have htop installed on few of the OpenVZ CentOS 5.5 Linux servers at work.

I looked online but unfortunately I couldn’t find any rpm pre-built binary packages. The source rpm package I tried to build from dag wieers repository failed as well, so finally I went further and decided to install htop from source

Here is how I did it:

1. Install gcc and glibc-devel prerequired rpm packages

[root@centos ~]# yum install gcc glibc-devel

2. Download htop and compile from source

[root@centos src]# cd /usr/local/src
[root@centos src]# wget "http://sourceforge.net/projects/htop/files/htop/0.9/htop-0.9.tar.gz/download"
Connecting to heanet.dl.sourceforge.net|193.1.193.66|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 418767 (409K) [application/x-gzip]
Saving to: "download"

100%[======================================>] 418,767 417K/s in 1.0s
2011-07-22 13:30:28 (417 KB/s) – “download” saved [418767/418767]

[root@centos src]# mv download htop.tar.gz
[root@centos src]# tar -zxf htop.tar.gz
[root@centos src]# cd htop-0.9
[root@centos htop-0.9]# ./configure && make && make install

make install should install htop to /usr/local/bin/htop

That’s all folks! , now my OpenVZ CentOS server is equipped with the nifty htop tool 😉

Some Helpful Subversion (SVN) general repository managing commands when you have to deal with Subversion on Debian Lenny servers

Friday, April 2nd, 2010

When I started with subversion it was a bit chaotic for me to grasp the subversion repository software basics.
Since I know there are many other people like me who are a novice into suversion I decided to post few of the
life saving (vital) subversion commands, I learned and use quite often this days.
This post should be considered as a very very overview of subversion commands. For more information please check, The subversion red-bean book here .
So here we go:
1. First To create repository after installing subversion you need to execute something similar to:

debian-server:~# svnadmin create --fs-type fsfs /path/to/repos/repo

In the above example /path/to/repos is actually the path to where you store the svn repositories, and repo is actually the repository name.
By the way note that by default svnadmin would create the repository in the fsfs database format, even if you skip the,
fsfs option.2. Let’s say you want to import some code into the newly created repository located in /path/to/repo via the local filesystem.
Here is how:

# imports in the subversion repositorydebian-server:~# svn import -m "importing directory in svn over local filesystem"
~/directory_to_import/ files:///path/to/repos/repo/trunk

In the forementioned example the, -m and the following text: “importing directory in svn over local filesystem” is for description of the importing data,
the ~/directory_to_import/ is the directory you prefer to import into the local repository, the code left,
files:///path/to/repo/trunk specifies that you want to import the data into the repository subdirectory “trunk”.

Then again let’s assume that you want to achieve a file import into a newly created repository through ssh + the apache mod_dav_svn

It’s pretty easy the above should be changed to:

debian-server:~# svn import -m "importing directory in svn over mod_dav_svn e.g. (svn+ssh)"
~/directory_to_import/ svn+ssh://user@host/path/to/repos/repo/trunk
of course it preliminary that you input a proper user and host or ip address as you have previously configured the mod_dav_svn, then again svn+ssh specifies the protocol type.

Now as we have imported our program source code into the repository, next it’s important to checkout the code to have a current copy of the source code.
3. To checkout code already existing in some repository in your subversion server via (svn+ssh) protocol, you need to execute some command similar to:

debian-server:~# svn co svn+ssh://user@host/path/to/repos/repo/trunk ~/checkout_into_directory/

Here again as a first protocol argument (svn+ssh://) it’s necessery to enter path/to/repos/repo/trunk and as a second argument to gsvn (the subversion command line client interface) we put ~/checkout_into_directory/ , it’s a nice idea to to create the checkout_into_directory beforehead.

Now if we have to checkout the code after we’ve been logged in the system and the repository database is locally stored on the same server as we are, we have to execute:

debian-server:~# svn co files://path/to/repos/repo/trunk ~/checkout_into_directory/

Take a note that in the example above I use the root user but possibly you would choose a non-privileged user, therefore you should have properly set both physical user account permissons on the subversion repository database (e.g. chown your /path/to/repos/repo/ and put your local user into the proper /etc/groups).

Another truly precious command that you will probably need to use on daily and hourly basis would probably be:
4. The listing of repository content cmd, in order to do that while locally logged on the server with the svn repository execute:

debian-server:~# svn list file:///path/to/repos/repo/trunk

I believe the above command is self-explanatory enough, in case if you plan to do file listing within the svn repository over (ssh+svn) here is how:

debian-server:~# svn list --verbose svn+ssh://user@host/path/to/repos/repo/trunk

Again, I won’t take the time to explain since the logic in the syntax is equal to the one exhibited beforehead.
5. Another handy thing to do with your subversion repository content after checkout is the subversion source repository update

the svn update The checkout will enable you to always synchronize your ~/checkout_into_directory to the latest stable version of the code within your svn repository.

So after the first checkout it would be good idea to use svn update and update your repository project source tree.
So here is how:

debian-server:~# svn update ~/checkout_into_directory/

So now as I have shown most basic operations with subversion, Lest important to show you is
6. How to delete source from a repository in subversion.

In order to delete some part from your subversion repository project source from the local filesystem use:
debian-server:~# svn delete files:///path/to/repos/repo/track/some_directory

This command would completely erradidacate some_directory from your example repo. Yet if you desire to delete a file specify a file instead of the some_directory

Now to accomplish the same delete operation via (svn+ssh) execute something like:

debian-server:~# svn delete svn+ssh://user@host/path/to/repos/repo/track/some_directory

Once again I won’t bother to explain the above example code, cause I believe it’s clear enough for everybody to understand.

7. To reverse your project code to some stable release of your source code existing in the repository you should use something like:

debian-serve~:# svn checkout -r 4 files:///path/to/repos/repo/trunk
This would checkout the project source to it's 4th release from the repository: repo

8. To commit code with changes in your subversion repository use a command like:

debian-server:~# svn commit -m "Some description text" some_directory/

The svn command line interface is also capable of svn copy and svn rename in order to either,
copy or rename commited source, however I won’t get into details on that just experiment and you’ll quickly master them.
9. Now one last thing I’m gonna tell you about is the subversion svn info command and svn status . This really useful command should be used to check information on your source tree after you have either checked it out or have used svn update to have the latest copy of it. This is an absolute necessity.

Here is how to check the information assigned about the version release and some other useful info for your source tree.

debian-server:~# svn info ~/check_into_directory
or you might type svn info without arguments as well
debian-server:~# svn info

Yet another useful one on project status is:

debian-server:~# svn status