renaming non-ascii file names

Mon, 08/14/2017 - 12:52 — sandip

Below script replaces non-ascii character with an underscore in file/folder names.

Before running the below script, cd to the folder with files that need renamed first.

#!/bin/bash
# rename2ascii.sh
# Replaces non-ascii character with an underscore.

function rename_to_ascii() {
  echo -n "$1" | \
  perl -ne '$new = $_; if($new =~ s/[^[:ascii:]]/_/g) {
    print("Renaming $_ to $new\n");
    rename($_, $new);
  }'
}

export -f rename_to_ascii

find -depth -exec bash -c 'rename_to_ascii "$0"' {} \;

sendmail dh key too small

Tue, 08/04/2015 - 21:17 — sandip

Logjam broke Sendmail?

Generate new DH keys file:

cd /etc/pki/tls/certs
openssl dhparam -out dhparams.pem 2048

Edit sendmail.mc:

define(`confDH_PARAMETERS';,`/etc/pki/tls/certs/dhparams.pem')dnl

Update sendmail.cf and restart:

cd /etc/mail
make
service sendmail restart

Refer to https://weakdh.org/sysadmin.html for other service fixes.

File Lock Contention

Thu, 04/30/2015 - 12:13 — sandip
(via blog.nexcess.net)

Great step by step explanation of investigating file lock contentions, specifically when dealing with php and sessions.

Delete All Oracle User Objects

Fri, 04/04/2014 - 11:04 — sandip

If you don't have system level access, and want to clean your oracle schema, the following sql will produce a series of drop statments, which can then be executed. Not all of them will execute - if you drop with cascade, dropping the PK_* indices will fail.

select 'drop '||object_type||' '|| object_name||  DECODE(OBJECT_TYPE,'TABLE',' CASCADE CONSTRAINTS;',';9;)from user_objects;

Confirm with:

select * from user_objects

Purge out the recyclebin if not needed:

purge recyclebin;

Resolve partial lvm logical volume

Wed, 03/19/2014 - 10:51 — sandip
  1. Server rebooted during lvm snapshot and put volume group in inconsistent state.
    # vgs
  VG   #PV #LV #SN Attr   VSize   VFree
  vg0    1   3   0 wz--n- 229.66G 9.66G
  vg1    2   3   0 wz-pn- 279.46G 9.46G

    Note the "p" attribute in vg1 which put the group in partial mode.

  2. To resolve, I first vgreduce then re-created the volume group from a good known archive:

memcached on centos

Sat, 12/07/2013 - 13:35 — sandip

Below is a summary of steps taken to get memcache support for php:

1. Install memcached and php-pear for pecl support to download and compile memcache for php.

    yum --enablerepo=epel install memcached php-pear
    pecl install memcache

2. Configure memcached (/etc/sysconfig/memcached) and start it up:

    chkconfig memcached on
    service memcached start

3. Create /etc/php.d/memcache.ini file with the below contents:

    extension=memcache.so

4. Reload apache and/or php-fpm processes:

    service httpd reload
    service php-fpm restart

5. Check with "php -i" or "phpinfo()" that memcache support is enabled.

6. Memcached listens to port 11211 by default. Status and stats:

    memcached-tool localhost:11211 display
    memcached-tool localhost:11211 stats

Adding utf-8 Byte Order Mark to file

Thu, 09/19/2013 - 15:57 — sandip

#!/bin/bash
# add_bom.sh

FILE=$1

check() {
  if (( e = "$1" )); then
    echo "ERROR: $ACTION failed!!"
    exit $e
  else
    echo "INFO: $ACTION Success!!"
  fi
}

set_bom() {
    ACTION="Setting utf-8 BOM"
    # Detect if BOM already exists
    BOM=`sed -n '1{/^\xEF\xBB\xBF/p}'; $FILE`
    if [ -z $BOM ]; then
        [ -f "$FILE" ] && /usr/bin/vim -e -s +"set bomb|set encoding=utf-8|wq" $FILE
        check $?
    fi
}

set_bom

To run, give the script the file name as argument:

sh ./add_bom.sh /path/to/file

Test the file via xxd, if the mark is present, the first line should output as below:

$ xxd </path/to/file
0000000: efbb bf...

OpenSSH SFTP chroot() with ChrootDirectory

Tue, 06/11/2013 - 10:20 — sandip
(via www.debian-administration.org)

The upcoming version of OpenSSH (4.8p1 for the GNU/Linux port) features a new configuration option : ChrootDirectory. This has been made possible by a new SFTP subsystem statically linked to sshd.

sandboxed Python development environments with pythonbrew

Thu, 05/30/2013 - 11:13 — sandip
(via suvashthapaliya.com)

As developers get started with a project, they prefer having total control over the development environment, as much as possible to perfectly match the deployment environment. The other strong requirement, especially in a team with mutiple developers would be to work on exactly replicated environments.

MySQL Database Restore

Wed, 05/29/2013 - 23:11 — sandip

I've had to do mysql database restores based on lvm snapshot backups, one too many times... from damaging sql statements. So here is some notes from the last one for future reference:

* Prior to restore, make sure no writes are coming to the corresponding db schema.

* Restore the database schema from the daily archived snapshot:

service msyqld stop
mv /var/lib/mysql/db1 /var/lib/mysql/db1_old
cd /var/lib
tar -xzf /opt/bak/db/latest_db.tgz mysql/db1
tar -xvzf /opt/bak/db/latest_db.tgz mysql/snapshot.log
service mysqld start

Note: The above process is for database schema with MyISAM tables only, so just restoring the corresponding db folder was sufficient.

The snapshot.log file has the status of the master/slave replicated positions shown via "show master status\G" and "show slave status\G". Additionally, I also pipe in the file sizes of the mysql bin logs at the time the lvm snapshot was taken.

Once the database has been restored from the daily snapshot archive, begin restoring from the binary logs to bring the data up to date.

* Extract from bin log using position recorded in the snapshot.log file:

mysqlbinlog --database db1 --start-position=XXXXXX mysql-bin.[0-9]* >db1.sql

* Edit the db1.sql file and remove the unwanted transaction.

Note: If you do not have exact positions, you could playback the logs with an approximate start and end date in the below format:

--start-datetime="YYYY-MM-DD HH:MM:SS" --stop-datetime="YYYY-MM-DD HH:MM:SS"

* However, If duplicate error occurs, change "INSERT" to "INSERT IGNORE" or "REPLACE" to overwrite:

perl -pi -e 's/^INSERT /INSERT IGNORE /g' db1.sql

* Check to make sure that no other database schema is being used, if found you will have to remove those lines as well if playing back just a single schema:

grep -i "^use" db1.sql

* Run import:

mysql db1 <db1.sql

With that import, the datbase schema should be caught up and restored back to it's current state with the unwanted transaction/s removed.

Comment