Category Archives: How-to

How to use PowerShell to (grep) Recursively Search for Text Within Files on Windows

Category : How-to

Get Social!

powershellThe thing I find most annoying with Windows is that it isn’t Linux. Let’s forget the argument of free software, the interchangeable GUIs, the security and everything else which constitutes the usual Linux vs. Windows argument and focus on things I use everyday in Linux which are missing in Windows. Two major things come to mind; tail for monitoring logs and grep which is the easiest way to find something in a file.

Not having grep, more specifically grep -r, is challenging at best and almost reason enough to avoid the platform entirely.

With the introduction of PowerShell, Windows has given us the grep functionality albeit with a much less finesse than the Linux equivalent. You have to pipe multiple commands together; one command to transverse the directories, and one command to look for the pattern within each file found.

Use the below command inside the directory you would like to perform the ‘grep’ and change [SEARCH_PATTERN] to match what you would like to match.

dir -Recurse | Select-String -pattern [SEARCH_PATTERN]

For example:

dir -Recurse | Select-String -pattern "Find Me"

As you can see, its nowhere near the memorable Linux command grep -r but at least its now possible go get similar behaviour in a Windows environment.

Other Windows grep Binaries

There are also various Windows binaries which can be used from a standard command prompt however I had limited luck with each one. The biggest issue was that they require dependencies such as .NET which are not usually installed in server environments.


Install Grails from Git on Windows

Category : How-to

Get Social!

grails-core-gradlew.bat-installGood news: installing Grails from source is easy-peasey on Windows! Before you get started, you’ll need to have git and a JDK installed.

You can download both git and the JDK from the following locations:

Once these are both installed, set the JAVA_HOME variable from a command prompt. You’ll need to locate the exact java version which is in your Program Files\Java folder as it changes with each version.

set JAVA_HOME=C:\Program Files\Java\jdk1.7.0_25

Move to the folder where you would like to deploy Grails.

cd c:\apps\

And run the git clone command to start downloading the source.

git clone git://github.com/grails/grails-core.git

Move into the folder which was created with the git clone command.

cd grails-core

Finally run the install command which will download any further dependencies and compile the application.

gradlew.bat install

And that’s it! I told you it was easy.


Export MySQL Database into Separate Files per Table

Category : How-to

Get Social!

mysql-logoI have recently been using git to check in an applications database. The database has many tables, some of which are populated with test data and created a fairly large file when exported. I noticed a few issues issues when checking these into git, namely that the large file was uploaded and saved in git as a single large file containing my changes and the other stuff which had not changed.

Instead of using this large file as one and checking it into git, breaking the file into several smaller files means that only the table which changed would be added to the git commit resulting in much smaller uploads.

The below code is a bash script which let’s you export, using mysqldump, all tables in a MySQL database to individual files. This will result in one file per MySQL table in the database. You will need to modify the following attributes:

  • [USER] – the username to use when connecting to the MySQL instance.
  • [PASSWORD] – the password for the above MySQL user.
  • [DATABASE] – the name of the MySQL database to export.
  • [BACKUP_LOCATION] – the location on the MySQL server where the SQL files will be created.
#!/bin/bash
GIT_MYSQL=/[BACKUP_LOCATION]
for T in `mysql -u [USER] -p[PASSWORD] -N -B -e 'show tables from [DATABASE]'`;
do
    echo "Backing up $T"
    mysqldump --skip-comments --compact -u [USER] -p[PASSWORD] [DATABASE] $T > $GIT_MYSQL/$T.sql
done;

Test Internet Speed from the Command Line

Get Social!

speedtest-netMatt Martez has kindly created and shared a Python script for interacting with speedtest.net for testing bandwidth to the internet from the command line. This is a great script which only has the dependency of Python – something which is available on most Linux distributions.

Grab the script with wget to the server you wish to test. Remember, this is to test the bandwidth to servers which are publically available on the internet – this does not test your internal server to server communication. See Iperf for details on testing LAN bandwidth.

wget -O speedtest-cli https://raw.github.com/sivel/speedtest-cli/master/speedtest_cli.py

Add the execute permission:

chmod +x speedtest-cli

And that’s all you need to start your first speed test! Run the speedtest-cli with the –simple switch to start your first basic speed test. The script will automatically choose the best server based on your location.

./speedtest-cli --simple
Ping: 10.331 ms
Download: 6.47 Mbit/s
Upload: 5.30 Mbit/s

You can use the –help switch to see all the options you have with the tool.

./speedtest-cli --help
usage: speedtest-cli [-h] [--share] [--simple] [--list] [--server SERVER]
                     [--mini MINI] [--source SOURCE] [--version]

Command line interface for testing internet bandwidth using speedtest.net.
--------------------------------------------------------------------------
https://github.com/sivel/speedtest-cli

optional arguments:
  -h, --help       show this help message and exit
  --share          Generate and provide a URL to the speedtest.net share
                   results image
  --simple         Suppress verbose output, only show basic information
  --list           Display a list of speedtest.net servers sorted by distance
  --server SERVER  Specify a server ID to test against
  --mini MINI      URL of the Speedtest Mini server
  --source SOURCE  Source IP address to bind to
  --version        Show the version number and exit

Setup Headless Dropbox Sync Client on Linux

Category : How-to

Get Social!

dropbox-logoDropbox is a cloud based file storage service which makes your files available from almost any internet connected device. You simply synchronize a folder with the service on each device and Dropbox keeps them in sync, automatically updating each folder as files are added and removed on each machine.

You can register for a free account which gives you a limited amount of storage to use – no strings attached. To register, visit Dropbox (Please note, that is my affiliate link which gives me a bonus if you sign up. If you’re not happy with this, you can simply visit dropbox.com and sign up. I’d be most grateful if you’d use the link :) ).

Dropbox offers a client to use on Windows, Mac OSX, Linux, iPhone and Android which you can download for free. These are all graphical interfaces and do not work for deployment on a headless server.

Thankfully, the Dropbox team have created an easily deployable Dropbox client which works without a desktop installed and can be managed by a Python script. To get started, download the Python script to your home directory on your headless server with wget.

cd ~
wget https://www.dropbox.com/download?dl=packages/dropbox.py -O dropbox.py

Give the script the permission to execute.

chmod +x dropbox.py

You can now use the Python script to download the Dropbox client. The client will be downloaded to your users home directory so make sure that you are logged in with the correct user.

./dropbox.py start -i

The client binaries will now be downloaded and installed. The next step is to register your account with the Dropbox client so that synchronization can begin. Start Dropbox for the first time with the Python script and you will be presented with a link. Paste the link into a web browser and login to your Dropbox account to grant access to the client. As soon as this process completes, your Dropbox client will begin synchronization.

./dropbox.py start
This client is not linked to any account... Please visit hhttps://www.dropbox.com/cli_link?host_id=10a5ce48e41d50f8135dd6fd55b70a91 to link this machine.

Once you have got all this working and the Dropbox client is synchronizing your first files, it’s time to add an init.d script so that the Dropbox client starts with your operating system. Things may differ here, depending on your Linux distribution. Add one of the below scripts to your init.d folder and substitute [USER] for the list of users who will use the client.

vi /etc/init.d/dropbox

Ubuntu/ Debian

#!/bin/sh
#dropbox service
DROPBOX_USERS="[USER]"

DAEMON=.dropbox-dist/dropboxd

start() {
   echo "Starting dropbox..."
   for dbuser in $DROPBOX_USERS; do
       HOMEDIR=`getent passwd $dbuser | cut -d: -f6`
       if [ -x $HOMEDIR/$DAEMON ]; then
           HOME="$HOMEDIR" start-stop-daemon -b -o -c $dbuser -S -u $dbuser -x $HOMEDIR/$DAEMON
       fi
   done
}

stop() {
   echo "Stopping dropbox..."
   for dbuser in $DROPBOX_USERS; do
       HOMEDIR=`getent passwd $dbuser | cut -d: -f6`
       if [ -x $HOMEDIR/$DAEMON ]; then
           start-stop-daemon -o -c $dbuser -K -u $dbuser -x $HOMEDIR/$DAEMON
       fi
   done
}

status() {
   for dbuser in $DROPBOX_USERS; do
       dbpid=`pgrep -u $dbuser dropbox`
       if [ -z $dbpid ] ; then
           echo "dropboxd for USER $dbuser: not running."
       else
           echo "dropboxd for USER $dbuser: running (pid $dbpid)"
       fi
   done
}

case "$1" in

   start)
       start
       ;;
   stop)
       stop
       ;;
   restart|reload|force-reload)
       stop
       start
       ;;
   status)
       status
       ;;
   *)
       echo "Usage: /etc/init.d/dropbox {start|stop|reload|force-reload|restart|status}"
       exit 1

esac

exit 0

Then add the execute permission and add it to the startup routine.

chmod +x /etc/init.d/dropbox 
update-rc.d dropbox defaults

RedHat/ Fedora/ CentOS

# chkconfig: 345 85 15
# description: Startup script for dropbox daemon
#
# processname: dropboxd
# pidfile: /var/run/dropbox.pid
# config: /etc/sysconfig/dropbox
#

### BEGIN INIT INFO
# Provides: dropboxd
# Required-Start: $local_fs $network $syslog
# Required-Stop: $local_fs $syslog
# Should-Start: $syslog
# Should-Stop: $network $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Start up the Dropbox file syncing daemon
# Description: Dropbox is a filesyncing sevice provided by dropbox.com
# This service starts up the dropbox daemon.
### END INIT INFO

# Source function library.
. /etc/rc.d/init.d/functions

# To configure, add line with DROPBOX_USERS="[USERS]" to /etc/sysconfig/dropbox
# Probably should use a dropbox group in /etc/groups instead.

[ -f /etc/sysconfig/dropbox ] && . /etc/sysconfig/dropbox
prog=dropboxd
lockfile=${LOCKFILE-/var/lock/subsys/$prog}
config=${CONFIG-/etc/sysconfig/dropbox}
RETVAL=0

start() {
   echo -n $"Starting $prog"
   if [ -z $DROPBOX_USERS ] ; then
      echo -n ": unconfigured: $config"
      echo_failure
      echo
      rm -f ${lockfile} ${pidfile}
      RETURN=6
      return $RETVAL
   fi
   for dbuser in $DROPBOX_USERS; do
      daemon --user $dbuser /bin/sh -c "~$dbuser/.dropbox-dist/dropboxd&"
   done
   RETVAL=$?
   echo
   [ $RETVAL = 0 ] && touch ${lockfile}
   return $RETVAL
}

status() {
   for dbuser in $DROPBOX_USERS; do
      dbpid=`pgrep -u $dbuser dropbox | grep -v grep`
      if [ -z $dbpid ] ; then
         echo "dropboxd for USER $dbuser: not running."
      else
         echo "dropboxd for USER $dbuser: running (pid $dbpid)"
      fi
      done
}

stop() {
   echo -n $"Stopping $prog"
   for dbuser in $DROPBOX_USERS; do
      killproc ~$dbuser/.dropbox-dist/dropbox
   done
   RETVAL=$?
   echo
   [ $RETVAL = 0 ] && rm -f ${lockfile} ${pidfile}
}

# See how we were called.
case "$1" in
   start)
      start
      ;;
   status)
      status
      ;;
   stop)
      stop
      ;;
   restart)
      stop
      start
      ;;
   *)
      echo $"Usage: $prog {start|status|stop|restart}"
      RETVAL=3

esac

exit $RETVAL

Change permissions and add to the startup routine.

chmod 0755 /etc/init.d/dropbox 
chmod 0644 /etc/sysconfig/dropbox
chkconfig dropbox on

Mount a Windows/ Samba Share in Linux

Category : How-to

Get Social!

Linux penguinGiven the amount of trouble you can have getting Linux and Windows to play nicely together, you may be surprised to find out that it’s easy to mount either a Windows file share on Linux. You can also use the same method to mount a Samba share.

This article assumes you already have a Windows or Samba share set up which is protected by a username and password. The user could be part of a domain or local to the machine which presents the storage.

The first step is to create a credentials file which will contain the username and password for accessing the file share. This needs to be in a protected area so that the details cannot be viewed. For this example, I will put the file in the root home directory.

vi /root./smbcreds

Add the following details, substituting the below details

  • [DOMAIN] – this is the domain for the file share user. This may not be required in all environments.
  • [USER] – this is the user name for the file share.
  • [PASSWORD] – this is the password of the above user.
username=[DOMAIN]\[USER]
password=[PASSWORD]

For example:

username=JAMESCOYLE\james.coyle
password=mypassword

Before continuting with the configuration, we need to make sure that the required packages are installed. On Ubuntu, you can use the apt-get command. Use your distributions package manager for other Linux distributions.

apt-get install cifs-utils

The next step is to create a folder where you will mount the Windows or samba share to in Linux. I will use the mount point /mnt/winshare.

mkdir /mnt/winshare

Now open fstab and add a new row at the bottom of the file to mount the remote share to the local folder. Add the below line to your fstab and substitute the values for your environment.

  • [SHARE_LOCATION] – is the remote host name/ DNS and shared folder name.
  • [MOUNT_LOCATION] – is the local path which will be used as the mount point for the Windows or Samba share.
  • cifs – is the type of mount to use. For older Linux systems you may need to use smbfs.
  • [CREDENTIALS_FILE] – is the location of the local credentials file.
//[SHARE_LOCATION]  [MOUNT_LOCATION]  cifs  credentials=[CREDENTIALS_FILE],iocharset=utf8,uid=1001,_netdev,file_mode=0770,dir_mode=0770 0 0

For example:

//windowsbox/sharename  /mnt/winshare  cifs  credentials=/root/.smbcreds,iocharset=utf8,uid=1001,_netdev,file_mode=0770,dir_mode=0770 0 0

Finally we need to create the mount which will use the details in fstab. Change [MOUNT_POINT] to match the mount point you created.

mount [MOUNT_POINT]

For example:

mount /mnt/winshare

Visit our advertisers

Quick Poll

How often do you change the password for the computer(s) you use?

Visit our advertisers