Perl

Perl Scripts and tricks...

renaming non-ascii file names

Below script replaces non-ascii character with an underscore in file/folder names.

Before running the below script, cd to the folder with files that need renamed first.

#!/bin/bash
# rename2ascii.sh
# Replaces non-ascii character with an underscore.

function rename_to_ascii() {
  echo -n "$1" | \
  perl -ne '$new = $_; if($new =~ s/[^[:ascii:]]/_/g) {
    print("Renaming $_ to $new\n");
    rename($_, $new);
  }'
}

export -f rename_to_ascii

find -depth -exec bash -c 'rename_to_ascii "$0"' {} \;

Extract IP address

One liner with grep to extract the IP addresses from a file.

grep -Eo '([0-9]{1,3}\.){3}[0-9]{1,3}' /path/to/file

The "-o" option prints only the matched parts.

One liner with perl:

perl -ne 'print if s/.*((\d{1,3}\.){3}\d{1,3}).*/\1/' /path/to/file

munin-node and postgresql plugins setup

While setting up munin to monitor postgresql, I was getting "[DBD::Pg not found, and cannot do psql yet]" when running `munin-node-configure --suggest | grep postgres`.

I confirmed that the rpm package "perl-DBI-1.52-2.el5" was indeed installed.

However, when I ran a test against the module, it failed with:

# perl -MDBD::Pg -e 1
Can't load '/usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi/auto/DBD/Pg/Pg.so' for module DBD::Pg: libpq.so.4: cannot open shared object file: No such file or directory at /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/DynaLoader.pm line 230.
at -e line 0
Compilation failed in require.
BEGIN failed--compilation aborted.

On checking the library, it returned with "libpq.so.4 => not found":

# ldd /usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi/auto/DBD/Pg/Pg.so
linux-vdso.so.1 =>  (0x00007fffb60bb000)
libpq.so.4 => not found
libc.so.6 => /lib64/libc.so.6 (0x00007fa36d2c2000)
/lib64/ld-linux-x86-64.so.2 (0x00007fa36d845000)

Indeed libpq.so.4 was missing since postgresql90-libs was installed which only includes "libpq.so.5".

To get libpq.so.4, "compat-postgresql-libs" package needed to be installed.

Once installed the perl module test passed and I was able to get the munin plugins linked using:

# munin-node-configure --shell | grep postgres | sh

IP range to CIDR conversion

I've often had to convert IP range with netmask to a CIDR notation. Below is a quick perl script to help with the conversion:

#!/usr/bin/perl -w
# range2cidr.pl

use Net::CIDR;
use Net::CIDR ':all';

if (@ARGV == 0) {
  die "Usage Example: $0 192.168.0.0-192.168.255.255 \n";
}

print join("\n", Net::CIDR::range2cidr("$ARGV[0]")) . "\n";

Installing Bugzilla on an ISPConfig site

These are some notes I had taken down during the installation of Bugzilla on a server with ISPConfig hosting control panel:

Requirements:

# yum install mysql-devel gd-devel libpng-devel libjpeg-devel freetype-devel libdbm-devel
# rpm -e mod_perl #else mod_perl2 does not install.

Create Site:

Site was created via the ISPConfig control panel with "web6_bugs" as the admin user and "web6" as the group.

Install:

$ cd /var/www/web6
$ wget http://ftp.mozilla.org/pub/mozilla.org/webtools/bugzilla-3.0.tar.gz
$ tar -xvzf buzilla-3.0.tar.gz
$ mv buzilla-3.0/* web
$ ./checksetup.pl --check-modules
# perl -MCPAN -e 'install AppConfig' #had to force install.
# perl -MCPAN -e 'install Bundle::Bugzilla'
# perl -MCPAN -e 'install GD'  #requires libpng-devel, libjpeg-devel, freetype-devel
$ ./checksetup.pl #Edit the localcofig with the correct database settings.
$ ./checksetup.pl #Re-run anytime if needed, specifically if the file permissions are not correct.
$ chgrp -R apache . #Had to make all files belong to the apache group after re-running checksetup.pl .

Apache with mod_perl need the below directives in httpd.conf:

    PerlSwitches -I/var/www/web6/web -w -T
    PerlConfigRequire /var/www/web6/web/mod_perl.pl

Runs a lot faster, but is a memory hog. Suggested to turn off KeepAlive in apache when running mod_perl.

Crontab Entry:

# Bugzilla
5 0 * * * web6_bugs cd /var/www/web6/web; ./collectstats.pl
55 0 * * * web6_bugs cd /var/www/web6/web; ./whineatnews.pl
*/45 * * * * web6_bugs cd /var/www/web6/web; ./whine.pl

Howto check and install missing perl modules...

  • Check if module is installed. Errors mean missing module.
    # perl -MModule::Name -e 1
    
  • See documentation of the module if installed.
    # perldoc Module::Name
    
  • Open CPAN shell:
    # perl -MCPAN -e shell
    
  • To reconfigure the shell if needed.
    cpan>o conf init
    
  • Install an available module.
    cpan> install Module::Name
    
  • Force install if test fails.
    cpan> force install Module::Name
    
  • To manual install perl modules.
    # perl Makefile.PL
    # make
    # make test
    # make install
    

Web Client Programming with Perl - Automating Tasks on the Web

An excellent free ebook which imparts knowledge on how the Web technically works and how to write software that is more flexible, dynamic, and powerful than the typical web browser using perl and perl tk. The goal is to give you the ability to retrieve, manipulate, and redistribute web-based information in an automated fashion.

Server Environment Variables

One of the methods that the web server uses to pass information to a cgi script is through environmental variables. These are created and assigned appropriate values within the environment that the server spawns for the cgi script. They can be accessed as any other environmental variable, like with %ENV{'VARIABLE_NAME'} (in Perl). Many of them, contain important information, that most cgi programs need to take into account.

To get a list of such environment variables, use the below script:

#!/usr/bin/perl
foreach $value(keys %ENV) {
  print "$value: $ENV{$value}\n";
}
exit;

Perl Compatible Regular Expressions Manual

PCRE was originally written for the Exim MTA, but is now used by many high-profile open source projects, including Python, Apache, PHP, KDE, Postfix, Analog, and nmap. Other interesting projects using PCRE include Ferite, Onyx, Hypermail, and Askemos.

CGI:IRC

CGI:IRC - is a Perl/CGI program that lets you access IRC from a web browser, it is designed to be flexible and has many uses such as an IRC gateway for an IRC network, a chat-room for a website or to access IRC when stuck behind a restrictive firewall.

Comment