Entries Tagged as 'bash'

Howto: Mass Rename File Extensions in Linux/BASH

Just recently I had a bunch of HTML files (files with a .html extension) that I wanted to rename to .cfm so that they would be interpreted by Railo. To do this, I created a quick BASH loop and ran it directly in the command line:

for i in *.html;
do mv $i `basename $i html`cfm;

The semi-colons at the end of the first two lines will tell bash that you want to enter more commands, and BASH will typically respond by giving you a new line with a greater-then symbol in front of it. So, the output of your loop will look something like this:

myserver $ for i in *.html;
> do mv $i `basename $i html`cfm;
> done
myserver $

The first line creates a "for" loop for every extension you want to change. In my case, since I wanted to change all files with the .html extension, I used .html. The second command creates a subshell (using the ` marks) and uses the basename program to remove the previous "html" entension. The subshell result is then passed back and used in our move command. For example, if you simply typed:

$ basename myfile.html html

So the subshell is returning the file name plus a dot. Once that happens the second move command turns into this:

$ do mv $i myfile.cfm

... which is what gets looped over as many times as we need it to in order to rename all the files!

Hope this helps!


BASH Execute a Command in a Variable - with Quotes!

Just recently I encountered a situation where I was using a BASH script to evaluate some input and then pass a boat-load of parameters to another executable. It just so happened that one of the parameters I was pasisng was a variable that had a space in it. So... --myvar "my spacey attribute".

I Love BASH!This command would work great when I ran the command myself, but as soon as I tried to execute the command in my BASH script, the blasted thing would not evaluate. The executable I was trying to pass the attribute with spaces totally ignored my quotes around the attribute value and evaluated my spaces as the beginning of another attribute. It was frustrating to the extreme! I tried all kinds of different quotation combinations. I re-wrote the same bit of code several different times in several different ways all with the same crappy results.

After much Google hunting, I came accross a post where someone suggested to another to use "eval" to correct a problem he was having with a parsed command (similar to what I was doing, but still quite different). The suggestion was shot down as being "too much" for that particular case, but I had never used "eval" before so I looked it up, and turns out it does exactly what I was needing and then some! "eval" is built int BASH so if you have BASH, you have eval. Here's my example...

Using this:

myCommand="$myExecutable --mode install ";
myCommand+="--directory /path/to/mydir/ ";
myCommand+="--password 'my password' ";

This did NOT work:

# now run the command

but this DID work:

# now run the command
eval $myCommand;

So, if you ever find yourself trying to figure out why your command isn't parsing correctly and pulling your hair out over it, try the "eval" command. I hope this helps someone and saves you the hours of frustration it would have saved me!!

scp "command not found" CentOS 6 / Ubuntu 12

Computer Frustration

I run Ubuntu 12.04 64-bit as my local Desktop OS. I've run the "SCP" command multiple times to multiple machines from my own PC when I'm moving files around, but for some reason when I tried to SCP, I got the following error:

dream@dream-C32A-H9:~/Downloads$ scp myfile.zip root@myserver.viviotech.net:/root
root@myserver.viviotech.net's password: 
bash: scp: command not found
lost connection

Okay... so that doesn't make any sense. I use SCP all the time. What is *really* going on here? To find out, I ran SCP in verbose mode, whcih you can do by adding the "-v" switch to your SCP command:

debug1: Authentication succeeded (password).
Authenticated to myserver.viviotech.net ([]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: Sending environment.
debug1: Sending env LANG = en_US.UTF-8
debug1: Sending command: scp -v -t -- /root
bash: scp: command not found
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: client_input_channel_req: channel 0 rtype eow@openssh.com reply 0
debug1: channel 0: free: client-session, nchannels 1
debug1: fd 0 clearing O_NONBLOCK
debug1: fd 1 clearing O_NONBLOCK
Transferred: sent 2048, received 2104 bytes, in 0.0 seconds
Bytes per second: sent 60339.9, received 61989.9
debug1: Exit status 127
lost connection

Ah... see these two lines:

debug1: Sending command: scp -v -t -- /root
bash: scp: command not found

So, we see that it was the REMOTE server that's complaining about the "command not found". Well, that should be easy enough to fix. I SSH'd over to the remote server and I was going to install SCP on it but I couldn't remember what package SCP was in, so I looked for it using YUM (since the remote machine was a CentOS machine):

[root@myserver ~]# yum provides */scp
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: centos.mirror.facebook.net
 * extras: ftp.osuosl.org
 * updates: mirrors.cat.pdx.edu
r1soft/filelists                                         |  14 kB     00:00     
updates/filelists_db                                     | 2.9 MB     00:01     
openssh-clients-5.3p1-81.el6.x86_64 : An open source SSH client applications
Repo        : base
Matched from:
Filename    : /usr/bin/scp

There we go. So the SCP command is provided by openssh-clients. I installed that by running the following:

yum -y install openssh-clients

Then I ran my SCP command again, and my file was uploaded without a hitch!

Hope this helps anyone else experiencing the same issues.

yum "Couldn't resolve host 'mirrorlist.centos.org'" for CentOS 6

I ran into an issue recently where I could ping URL's just fine, but when I ran the "yum update" command yum could not resolve anything. I would get error messages like the following:

[root@viviotech ~]# yum -y update Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=6&arch=x86_64&repo=os error was 14: PYCURL ERROR 6 - "Couldn't resolve host 'mirrorlist.centos.org'" Error: Cannot find a valid baseurl for repo: base

DNS and BindAfter a bit of digging, and absolutely no help from my friend Google, I asked one of the Vivio technicians about it and he said he'd seen it on other servers - specifically servers with VirtualMin.

Take a look at the resolve.conf file:

nameserver nameserver nameserver

See how VirtualMin adds the address first? Apparently YUM will ONLY check the very first entry in the /etc/resolve.conf file when it looks for the server to resolve IP Addresses. In this case, the local DNS resolver was not configured to use forwarders, so the YUM process would ask the local DNS server where to find "mirrorlist.centos.org", and when the local resolver didn't know, YUM would simply report an error instead of looking at the other resolvers in the /etc/resolve.conf list. This is why I could ping URL's just fine, but YUM could not find them.

The solution to this was to simply place the entry at the botton of the /etc/resolv.conf file, like so:

nameserver nameserver nameserver

After that, eveything worked great.

IMPORTANT: The above examples of the /etc/resolv.conf file use the IP Addresses of the Google Public DNS.

The tricky little "httpd -M" command

A project I'm doing requires that I write a BASH shell script that checks what modules are installed in Apache (to make sure some requirements are met before an install is attempted), however, I had a HECK OF A TIME figuring out how to script the checking of the httpd -M command (the command that shows the installed Apache modules). I was tring to run the following on a CentOS 5 server:

# httpd -M | grep -c perl

And the results of the command were not being piped to grep. Instead, the results were being sent to the screen. Irritating! There's clearly a pipe character there. I kept thinking I was writing the command wrong or something. I tried the command on my Ubuntu desktop, and lo and behold it worked just fine:

$ sudo apache2ctl -M | grep -c perl

apache2: Could not reliably determine the server's fully qualified domain name, using for ServerName
Syntax OK

Broken ApacheThe "1" was the result I was looking for, because I knew mod_perl was installed.

Long story short, for some reason the CentOS 5 version of Apache sends the "httpd -M" results to stderr, rather then stdout. I have no freaking idea why it does this, because that's not error data. It seems like a bug in the HTTPD executable. The end result was that I needed to write my command as follows:

# httpd -M  2>&1 | grep -c perl


The 2>&1 sends the stderr data to stdout which is what the pipe character sends to the grep command.

How annoying. For the record:

# httpd -v

Server version: Apache/2.2.3
Server built:   Oct 20 2011 17:00:12

Hope this helps someone avoid the frustration and confusion I went through finding it.