RSS
 

Archive for March 30th, 2006

Installing Perl modules on UNIX w/o root permissions

30 Mar

Installing Perl modules on a UNIX virtual server when you don’t have root or administrator privledges.

Before you begin: In order to install modules, you have to have SSH or Telnet access to the server. If you don’t have this access (or you don’t know what SSH/Telnet means) then you’ll probably have to get your system administrator to install modules for you.

I recently wanted to install the DBD::CSV module on my virtual host server (UNIX FreeBSD) and found that it was more challenging than I would have liked. I’m not positive I have the best approach, but it did work for me in the end. Following are the steps that I had to take to get the module and all those on which it depends installed. The primary problem is that on the virtual server, I don’t have root/administrator privledges. So, I have to install Perl modules in an alternate directory. The scripts have to be altered to point to that directory as well.

To install my beloved DBD::CSV, I notice that I also have to have all of the following modules installed and working:

DBI
File::Spec
Text::CSV_XS
SQL::Statement

So, we will install those first, then finally install DBD::CSV.

First, I set up MCPAN. This is a Perl utility that automatically downloads, unpacks, compiles and installs Perl modules. Since I don’t have root, I was only able to utilize MCPAN for the downloading and upacking part. I still had to compile and install the modules manually. If you don’t have access to, or don’t want to bother with MCPAN, then you can just download the modules directly from http://www.cpan.org and unpack them yourself.

To run MCPAN, you enter something like this (this will start the process of installing DBI, the first required module):

perl -MCPAN -e 'install DBI'

The first time I ran this utility, I was automatically taken through an installation process. A directory ~/.cpan was created in my home directory and I was prompted with several questions. I simply used all the defaults. There was only one question that I had trouble with, which was something like “What is your favorite CPAN mirror.” This seemed to be asking where to look for downloadable modules. I wish I remembered what I put, but it took me several tries to find one that worked. I finally wound up entering a server in Japan, which I’m sure is not good. But it worked. You might check for possible server locations here: http://www.perl.com/CPAN

Anyway, after running that the first time, the DBI module installation files were saved in ~/.cpan/build/DBI-1.14. (in case you don’t know ~/ usually means your home directory), however the installation failed because it tried to write some stuff to the main Perl installation location and, again, I don’t have permission to write there. however, when you are manually installing modules, I do know that you can specify an alternate directory. So, I went into the ~/.cpan/build/DBI-1.14 directory and manually installed it like so:

cd ~/.cpan/build/DBI-1.14
perl Makefile.PL INSTALLDIRS=site INSTALLSITELIB=/usr/www/users/myaccount/cgi-bin/lib
make
make test
make install

When I installed it, I see that I got a few errors and such. Upon closer inspection, it seems that the POD documentation and other stuff is still trying to go in the main Perl library location. I ignore these errors, since it doesn’t seem to effect anything but the documentation. Obviously, you want to replace /usr/www/users/myaccount/cgi-bin/lib with the location on your server where you want to save the modules.

Anyway, now I try the File::Spec module:

perl -MCPAN -e 'install File::Spec'
cd ~/.cpan/build/File-Spec-0.82
perl Makefile.PL INSTALLDIRS=site INSTALLSITELIB=/usr/www/users/myaccount/cgi-bin/lib
make
make test
make install

That one was easy. I still got errors, but I just ignore them. Now I try Text::CSV_XS:

perl -MCPAN -e 'install Text::CSV_XS'
cd ~/.cpan/build/Text-CSV_XS-0.21
perl Makefile.PL INSTALLDIRS=site INSTALLSITELIB=/usr/www/users/myaccount/cgi-bin/lib
make
make test
make install

Same old thing. More errors. Moving along to SQL::Statement…

perl -MCPAN -e 'install SQL::Statement'
cd ~/.cpan/build/SQL-Statement-0.1016
perl Makefile.PL INSTALLDIRS=site INSTALLSITELIB=/usr/www/users/myaccount/cgi-bin/lib
make
make test
make install

Another one bites the dust. The final one, DBD::CSV is more tricky because when I tried the same thing, it doesn’t work. I can’t do the Makefile.PL part of it because it says I am missing some required modules. What is happening is that the installer is looking in the standard Perl path – however we have been installing all of our goodies to an alternate location and Perl doesn’t know about it. My solution is that I simply comment out the lines in Makefile.PL where it checks for all the required modules! That’s a crappy way to do it, but it works.

perl -MCPAN -e 'install DBD::CSV'
cd ~/.cpan/build/DBD-CSV-0.1024

Now I edit Makefile.PL and comment out these lines (about 20 lines down from the top or so)

# $ok &&= CheckModule('DBI', '1.00');
# $ok &&= CheckModule('Text::CSV_XS', '0.16');
# $ok &&= CheckModule('SQL::Statement', '0.1011');

That’s it. the rest is the same.

perl Makefile.PL INSTALLDIRS=site INSTALLSITELIB=/usr/www/users/myaccount/cgi-bin/lib
make
make test
make install

All done at last! I tried my script and it worked just fine. One adjustment that you might have to make to your script is to add the location of your libraries to your script. You do this by putting the location in a BEGIN clause in your script like so:

BEGIN {
unshift(@INC,"/usr/www/users/myaccount/cgi-bin/lib");
}

Put this at the top of your script before any other subroutines or anything. That tells your script to look there when loading modules. Since you use the “unshift” function, your path is added to the front of the list. If you would rather your script look in the regular places first, and look in your custom path last, then you can use “push” instead of “unshift.”

One final thing. I went into ~/.cpan/build/ and deleted all the files and folders. I get charged for disk space overusage, so no use keeping all those installation files, right?

 
No Comments

Posted in *NIX, Perl

 

File Permissions

30 Mar

This page explains how to set file permissions for the three most common type of web files: pages, scripts and data/config files.

To keep things simple, let’s make the following assumptions:

a. pages should be readable/writable by the owner and readable by the web visitor.
b. scripts should be readable/writable/executable by the owner and readable/executable by the web visitor.
c. data-config files should be readable/writable by the owner and readable/writable by the web visitor.
And also, lets use the following abbreviations:

- – - (or 0) = no permission
r – - (or 4) = read-only permission
rw – (or 6) = read/write permission
rwx (or 7) = read/write/execute permission

To change UNIX file permissions using your shell account (Telnet):

When changing file permissions on a UNIX server, there are three groups to which you assign permissions: owner, group, other. Owner is typically you. Group is typically all users with accounts on your server. Other is typically the web visitors. Using the chmod command on a UNIX server, you can set the permissions for each of those groups.

1. Log into your account and go to the directory where the files are located
2. Use the chmod command to change permissions like so:
2a. chmod 604 page.html
2b. chmod 705 script.cgi
2c. chmod 606 config-data.txt

To change UNIX file permissions using your FTP software:

1. Log into your account and go to the directory where the files are located.
2. Highlight the file that you want to change permission.
3. Locate the “file permission” or “chmod” command on your FTP software software (you may need to refer to the manual or help file)
4. There should be three groups. Each group should have either checkboxes or a selection for the permission type.
4a. set pages to rw- for the owner, no permission for the group, and r–for other
4b. set scripts to rwx for the owner, no permission for the group, and r-x for other
4c. set data/config files to rw- for the owner, no permission for the group, and rw- for other

 
No Comments

Posted in *NIX

 

common UNIX commands for webmasters

30 Mar

The following is a list of UNIX commands that I have found helpful when modifying web sites on the server. Most UNIX commands have many options and parameters which I have not listed here. Instead I have given examples of practical uses. For more complete information on most command, you can refer to the online manual by typing man [command] at the UNIX prompt. Some commands you can type [command] –help or [command] -?

Note, when I specify something in brackets like so: [filename] that is to indicate that you type in a filename or whatever. Do not include the brackets in your command.

Navigating UNIX:

/ (refers to the root directory on the server)
./ (the current directory that you are in)
../ (parent directory of your current directory)

pwd (shows what you current directory is – giving the full path)

ls (lists all the files in your current directory)
ls -al (lists filenames + information)
ls -alR (lists filenames + information in all subdirectories)
ls -alR | more (lists filenames + information in all subdirectories,
pausing when the screen become full)
ls -alR > result.txt (lists filenames + information in all subdirectories,
and ouputs the results to a file instead of the screen)
ls *.html (lists all files ending with .html)
ls -al /home/usr/bob/ (lists files + info for /home/usr/bob)

cd (changes you to a new directory)
cd images
cd / (changes you to the root directory)
cd /home/usr/images
cd .. (this goes back one directory)

Moving, Copying and Deleting Files:

mv [old name] [new name] (move/rename a file)

cp [filename] [new filename] (copy a file)

rm [filename] (delete a file)
rm * (delete all files in your current directory)
rm *.html (delete all files ending in .html
in your current directory)

Creating, Moving, Copying and Deleting Directories:

mkdir [directoryname] (creates a new directory)

ls -d */ (lists all directories within current directory)

cp -r [directoryname] [new directoryname] (copy a directory and all
files/directories in it)

rmdir [directoryname] (remove a directory if it is empty)
rm -r [directoryname] (remove a directory and all files in it)

Searching Files and Directories

find / -name [filename] -print (search the whole server for a file)
find . -name [filename] -print (search for a file starting with
the current directory)
find / -name [directoryname] – type d -print
(search the whole server for a direcory)

grep [text] [filename] (search for text within a file)

sed s/[oldtext]/[newtext]/g [filename] (searches file and replaces all occurances of [oldtext] with [newtext]

Viewing and Editing Files:

tail [filename] – view the tail end of a file, useful for checking the error log
when debugging a script

vi [filename] – opens a file using the vi text editor. you are a true geek if you use vi, however it’s fairly easy to use. (refer to the vi primer in this support forum)

Installing Software & Scripts

For downloaded ‘tar’ scripts, to un-tar and un-gz
tar -xvf [archive.tar] – extracts files from the tar archive ‘archive.tar’
tar -zxvf [archive.tar.gz] extracts files from the tar archive ‘archive.tar.gz’

Getting Server Information

which perl displays the path to perl

For viewing disk space.
du to view disk usage on server
quota to view your disk usage on server

whoami – displays your current username

uptime – displays how long the server has been up and some performance statistics

ps – displays running processes

top – (may be only available to admins) similar to windows task manager

kill -9 [process Id] – terminiates a running process (out of control CGI, etc). The process Id can be obtained using “ps”

File and Directory Permissions

There are three levels of file permission: read, write and execute. In
addition, there are three groups to which you can assign permission,
The file owner, the user group, and everyone. The command chmod followed
by three numbers is used to change permissons. The first number is
the permission for the owner, the second for the group and the third
for everyone. Here are how the levels of permission translate:

0 = — (no permission)
1 = –x (execute only)
2 = -w- (write only)
3 = -wx (write and execute)
4 = r– (read only)
5 = r-x (read and execute)
6 = rw- (read and write)
7 = rwx (read, write and execute)

I prefer that the group always have permission of 0. This prevents other
users on the server from browsing files via Telnet and FTP. Here are the
most common file permissions used:

chmod 604 [filename] (minimum permission for www HTML file)
chmod 705 [directoryname] (minimum permission for www directories)
chmod 705 [filename] (minimum permission for www scripts & programs)
chmod 606 [filename] (permission for datafiles used by www scripts)
chmod 703 [directoryname] (write-only permission for public FTP uploading)

Note that some systems use AFS filesystem and chmod does not behave as expected.You can sometimes identify AFS if the path that you are using begins like so /afs/path/to/files/ If your system uses AFS, then the following commands are used instead of chmod.

fs setacl [directory] [group] [access] (set file permissions)
fs listacl [directory] (list file permissions)

example:

fs setacl . httpd rliw (set read, lookup, insert, write to http for current dir)

Scheduling Tasks

You can schedule tasks to run automatically by using the UNIX cron command.
To use this, you create a text file with cron instructions, then process
this file. cron instructions are basically UNIX commands with extra info
about the time that they will run.

One important thing to note is that it is best to use full paths when
creating your cron file. As an example, create a file called mycronfile
and in it place one line:

0 1 * * * cp /usr/www/file.txt /usr/www/backup.txt

now at the command line, type the following:

crontab mycronfile

You have just scheduled an automated task! This task will run at the time
specified until you decide you want to cancel it.
There are six fields in this file. The first five represent the time that
the job will run. The sixth field is a UNIX command that will run at the
specified time. The above example will run every night at 1AM, at which
time it will copy a file.

Here is how the fields break down:

Field 1 | Field 2 | Field 3 | Field 4 | Field 5
Minutes | Hours | Day of Month | Month | Day of Week
(0-59) | (0-23) | (1-31) | (1-12) | (0-6)

You can enter a number in the field, a range of numbers, or an * to indicate all.
Here are a few more examples. These examples use the ls command, which would be
pretty useless. Note the time that it runs, though.

0 1 * * 1-5 ls (this would run every Monday-Friday at 1am)
0 1 * * 1,3,5 ls (this would run every Monday, Wednesday and Friday at 1am)
10 2 1 * * ls (this would run at 2:10am on the first of every month)
0 1 1 1 * ls (this would run at 1am on January 1 every year)

If you have a more complicated command that you want to run, it is sometimes
helpful to create a shell script and have that script run. You specify the
shell script as you would any UNIX command. For example:

0 1 * * * /usr/www/myscript

There are some other crontab switches that are useful:

crontab -l (lists your currently scheduled tasks)
crontab -r (delete all currently scheduled tasks)
crontab -e (directly edit your scheduled tasks)

Credits
——–

Originally created by J. Hinkle
Additional content provided by Dave Wojo

 
2 Comments

Posted in *NIX