Dropbox for Servers

I love dropbox. Let me say that again: I love dropbox! I’ve been using it for well over a year now and it’s the best file synchronizing / sharing service I’ve found (and I’m certainly not alone in that opinion). I use it on my laptop, my home desktop, my home server, my iPhone, my iPad, and with selective-sync, on my wife’s laptop. All of my important files (up to 50 GB worth which is what I pay for) are synced between all of those places all of the time. I don’t send 20 MB email attachments (which completely annoys me), I send links to download from dropbox. The list goes on and I’m now on a bit of a tangent.

Sever Use Case

I work on a lot of shared servers, such as for POVs (proof of value, benchmark) and most of them are Linux-based (I also sometimes use Amazon EC2 instances to teach APEX Classes). I found myself spending a lot of time transferring files to and from the server such as SQL scripts, AWR reports, SQL Monitoring reports, etc via scp (sometimes rsync). I didn’t want to setup my dropbox account on the server as well since it has all of my personal files too. Then I had a bit of an epiphany(and I’m sure I’m not the first one to think of this):

1. Create a 2nd, free dropbox account (2 GB of free space)
2. Setup that account on the linux server using the CLI version of dropbox located here. Dropbox has a formal, supported Linux client, but there were a number of package dependencies and specific versions that were more challenging to work around than it was worth.
3. Share a folder between my primary dropbox account and my 2nd dropbox account.

Since all my secondary dropbox account will see are the folders I share with it, anyone else that happens to be on the server won’t have access to my personal files. Whatever files I add to that folder on my laptop will almost instantly show up on the server and vice versa.

Security Caveat

Having co-authored a book on security (“Applied Oracle Security“), it’s clearly one of my primary interests. While dropbox does encrypt your files when stored on it’s server and does use TLS to encrypt them on the network, and they have policies and internal controls in place to prevent their own employees from decrypting them without authorization, it’s still possible. They store and can access the encryption keys. If the government compels them to decrypt your files, they can and will. So, if you’re storing your tax records or corporate intellectual property, I strongly encourage you to use some form of client-side encryption. Personally, I use a truecrypt file (which I mount as a disk) for super-sensitive stuff within dropbox. Actually, my whole dropbox folder sits on a truecrypt volume (50 GB) which prevents someone that steals my laptop from getting to all of my files, but this does nothing to stop dropbox employees from accessing them. I then create small (1-30 MB) truecrypt files / volumes within dropbox to encrypt anything that is super-sensitive before it leaves my laptop. So, my layers are Big Truecrypt Volume > Dropbox Folder > Little truecrypt volumes for any sensitive files.

Static Version of rlwrap for sqlplus

Quick Summary

If you don’t have root access to a machine, it can be challenging to compile rlwrap (a nice companion to sqlplus). You can compile it on another machine / VM, then use a utility called statifier to produce one binary that includes all of the required dependencies. If you want to just cut to the chase, you can download the version I compiled for OEL 5.6 x64 here.

How to Compile

If you don’t trust what I’ve compiled (and you probably shouldn’t if you care about security), here are the steps to compile your own version.

On a machine or VM that you control running a comparable Linux distribution:

1. Download and compile rlwrap.
2. Download and compile statifier.
3. Compile your static version of rlwrap:

   # statifier /usr/local/bin/rlwrap /tmp/rlwrap_static_x64

Copy the file to your locked-down server (where you don’t have root), then:

1. Put it in some logical directory such as ~/local/bin

2. chmod +x ~/local/bin/*

3. I create a symlink for it (but you could just rename it if you want):

   ln -s ~/local/bin/rlwrap_static_x64 ~/local/bin/rlwrap

4. Edit ~/.bashrc and add it to your path:

   export PATH=$PATH:~/local/bin
   alias sqlplus='rlwrap sqlplus'

5. source ~/.bashrc

More Details

You might be saying, “but Tyler, surely your sys admins trust you with root?”. No, they don’t, and I’m fine with that. I work on a lot of Exadata POVs where we have absurdly tight timeframes. The machines are managed by another group (a group which I have a tremendous amount of respect for). So, I never get root, but that also limits my liability to some degree if something should go wrong.

Yes, I know you can compile the required libraries (gnu readline and ncurses) and set compiler flags to point to different directories for these libraries, but something always goes wrong and I’m tired of hacking around with it.

A Few Linux Tips and Tricks

I feel I’ve neglected the “Linux” aspect of my blog, so here a few tips that I hope you find useful.

What port is the TNS Listener listening on?

sudo netstat -tlnp | grep tns
tcp        0      0 0.0.0.0:1521            0.0.0.0:*               LISTEN      15481/tnslsnr

I know this is a simple one, but it never hurts to review the basics.

Can I get to a particular port on a remote server?

netcat -w 3 -z -vvn 64.233.187.99 79-81
(UNKNOWN) [64.233.187.99] 81 (?) : Connection timed out
(UNKNOWN) [64.233.187.99] 80 (www) open
(UNKNOWN) [64.233.187.99] 79 (finger) : Connection timed out

This example is essentially running a port scan on google.com from port 79 to 81, waiting for 3 seconds for a port to time out. You can see that only port 80 is open. I use this constantly when troubleshooting connections between Application Servers and Databases, especially when firewalls are involved. This is much more meaningful than a simple ping. Note that netcat only accepts IP addresses, not host names, so you can ping a host first to get it’s IP address first. Also note that port scans may violate your own network policies and could be construed as a form of hacking, so use at your own risk and tell someone what you’re going to do before you do it. You can install netcat on Windows using Cygwin (the command is nc in cygwin).

Leave a remote session running without using VNC

I discovered the screen command (documented here) about a year ago and I now use it on a regular basis. For example, I did an APEX install on a remote server yesterday. I connected via SSH using PuTTY, typed screen, then started sqlplus and kicked off the install. I then typed ctrl+a d to disconnect from that screen session. I then exited SSH and closed PuTTY. Without screen, that would have killed my install. Since I started the install in a screen session, in continued on in the background. I could then SSH into the machine later in the day and type screen -r to reattach to the session. Trust me, if you use screen for a week, you’ll wonder how you lived without it.

Share the current directory over http

Add the following line to your .bashrc or .bash_profile file:

alias http='python -m SimpleHTTPServer 8000'

Source the file you added it to or logout / login for the change to be reflected. Now navigate to any directory and type webshare. Presto! You can now access that directory from a browser on port 8000.

When I read this tip here, I was simply blown away. It couldn’t be that easy, could it? Yep, it’s that simple and is one of the first aliases I create on a new system.

Sort folders by size

du --max-depth=1 /home/ | sort -n -r

Blink the lights on network card eth0

sudo ethtool -o eth0

Why in the world would you want to do this? Well, I was at a customer site in Pennsylvania last month assisting with (more observing really) a RAC install. Each of the 2 nodes had 9 network cards! This was a demo system shipped in by a hardware vendor, so the cards weren’t even in matching slots for the 2 nodes. We had no idea what card Linux had assigned eth0,1,2… to. I called Sergio Leunissen for a little help and within a few minutes he had the answer. Ethtool will continue to blink the lights on the card until you press ctrl+c, which made it easy for use to map their locations on paper and get the right cables for each network segment in the right card. Thanks Sergio.

For more information on Linux command-line techniques, take a look at “Guide to Advanced Linux Command Mastery” by Arup Nanda: Part 1 , Part 2.

Linux Package Management – The “Smart” Way

Intro
Anyone who has ever worked with Linux has run into the RPM dependency nightmare. You simply want to install RPM x, but it requires y and y requires z (it gets worse, but I will leave it at that). Package managers such as UP2DATE, APT, and YUM have made this significantly easier by handling the dependencies for you. About a year ago, I stumbled upon the Smart Package Manager and my Linux experience has never been the same. Why I love Smart:

• Smart can be configured with YUM, APT, UP2DATE, URPMI, directory of RPMs (and more) repositories
• It handles package dependencies, even if it needs to pull packages from several different sources
• It provides both GUI and Command Line interfaces

I’m a big Linux advocate, but my interest in Linux is really about the software I run on Linux, not Linux itself. I view Linux as a service that runs my software (mostly Oracle), and doesn’t get in my way. For me, any extra time I invest in configuring Linux is a waste. This post is all about spending less time messing with RPMs and more time with more important topics. This post is targeted at people running Linux in a development or demo environment (VMWare comes to mind), and NOT FOR PRODUCTION SYSTEMS. The 3rd party packages may invalidate your support agreements or de-stabilize your environment.

For this example, I downloaded Enterprise Linux (x86, 32-bit) from http://edelivery.oracle.com/linux It’s 100% source and binary compatible with Red Hat Enterprise Linux 4 and it’s free. Since it’s really RedHat 4, Update 4 with some bug fixes, we’ll point at those repositories, as well as some from CentOS 4.4. There are HowTos out there for many other distros, including Suse (my desktop distro of choice). For 64-bit EL, you’ll need to adjust all of the URLs and test, but the same concepts apply.

Run all commands from a bash shell as root. I’ve omitted all prompts and output to make copy-and-paste easier, so interpret each line as a new command.

Install Smart
mv /etc/yum.repos.d/ULN-Base.repo /etc/yum.repos.d/ULN-Base.repo.old # These repositories are not used
rpm -Uhv http://apt.sw.be/packages/rpmforge-release/rpmforge-release-0.3.6-1.el4.rf.i386.rpm # --httpproxy someproxy.com --httpport 80
yum install smart*

Add Local Repository
Copy all Enterprise Linux RPMs locally. In my case, they are on the host OS, accessed via VMWare shared folders at /mnt/hgfs/el-rpms. This will be our primary Smart channel. Make sure you adjust the path in this command based on the location your RPMs.
smart channel --add local-EL-rpms name="Oracle Enterprise Linux RPMs" manual=true type=rpm-dir path=/mnt/hgfs/el-rpms/ -y;

Add Remote Repositories
smart channel --add Dag-Wieers type=rpm-md priority=-5 baseurl=http://apt.sw.be/redhat/el4/en/i386/dag/ -y;
smart channel --add VA-Tech-Extras type=rpm-md priority=-5 baseurl=http://mirror.cs.vt.edu/pub/CentOS/4.4/extras/i386/ -y;
smart channel --add VA-Tech-Base type=rpm-md priority=-5 baseurl=http://mirror.cs.vt.edu/pub/CentOS/4.4/os/i386/ -y;
smart channel --add Dries type=rpm-md priority=-5 baseurl=http://ftp.belnet.be/packages/dries.ulyssis.org/redhat/el4/en/i386/dries/RPMS/ -y;
smart channel --add atrpms name="ATrpms Repository" type=rpm-md priority=-5 baseurl=http://dl.atrpms.net/el4-i386/atrpms/stable/ -y;

Notice I set the priority of all 3rd party repositories to -5, leaving the default priority of 0 for installed packages and the local RPMs. This will cause the UBL packages to take precedence whenever the same package is found in multiple places.

Command-Line Usage
Find all packages with PDF in their title
smart search pdf
Install the xpdf packge
smart install xpdf
Install a local or remote RPM, using the configured channels to fulfill required dependencies
smart install somefile.rpm
smart install ftp://someurl.com/somefile.rpm

Demo, Including GUI Usage
Click here for the full-size version
[splashcast GKBY3325TQ]

Advanced
If you are required to use a proxy server to access the internet, use the following commands to set the proxy server for smart (documented here):
smart config --set http-proxy=http://someproxy.com:80
smart config --set ftp-proxy=ftp://someproxy.com:80