Why to choose nfs over vboxsf for mounting host folders inside a VirtualBox VM!

Simple answer: SPEED

Long answer:

Most projects I work on a the moment are web related and I like my development environment to be as close to production as possible.

Therefore I set up a Debian Jessie some time ago and mounted my whole /Users/cb0/Sites directory into the VM. So I configured the shared folders inside the VirtualBox and used this code inside /etc/fstab inside the Debian.

Sites /Users/cb0/Sites vboxsf defaults,rw,uid=33,gid=33 0 0

This worked fine for custom php apps and even for symfony 1.4. Now I’m working on a symfony 2 project which seems to generate a whole bunch of cache files before starting to deliver the webpage.

This resulted in an average load time of 45s (yes seconds) which I could not bear for more than 2 pageloads. So I search for the bottleneck and found out that the creation of the cache files took too long. After reading some documentations I decided to try to mount the share over nfs instead of the vboxsf.

Here’s how to do it:

1. [Host] Create or open the file “/etc/nfs.conf” and insert the following line:

nfs.server.mount.require_resv_port = 0

2. [Host] Edit the /etc/exports file and configure your desired share folder like this:

/Users/cb0/Sites -mapall=501:20 -network 192.168.56.0 -mask 255.255.255.0

Replace “192.168.56” with the address you have chosen for your Host-only Adapter.

3. [Host] Run the following 3 commands:

> sudo nfsd update
> sudo nfsd checkexports
> showmount -e

The ‘showmount’ command will output something like this if everything went fine:

# Exports list on localhost:
/Users/cb0/Sites                    192.168.56.0

4. [GUEST] Install necessary software. (As mentioned I use debian jessie)

sudo apt-get install nfs-common

5. [GUEST] Add the mount infos into the file /etc/fstab

192.168.56.1:/Users/cb0/Sites /Users/cb0/Sites nfs soft,intr,rsize=8192,wsize=8192 0 0

This tells debian to mount the directory /Users/cb0/Sites located on my host machine into the exactly same directory on my linux. (Note: You have to create the folder inside the guest first.) I do this for convenience, you could mount it somewhere else.

6. [GUEST] Now mount this share.

sudo mount /Users/cb0/Sites

and you are done.

After these simple steps and no other tuning my sf2 web project load in about 580ms. This is more than 70times faster than before.
I also noticed a small speedup in my other web apps. Now I can relax, sit back and continue my work on this project.

Create symlinks inside a VirtualBox shared folder

While working inside debian, which is installed inside a VirtualBox and shares a folder with it’s host machine, I repeatingly got errors while creating symlinks. (I use “os x 10.7” as host and “debian wheezy” as guest os.)

Until now I accepted it but today I got tired of creating these symlink in the host system.

Here’s the solution I found:

1. Shut down the virtual machine. Debian in my case.
2. You need to close the VirtualBox GUI.
3. Type this into your preferred command line.

VBoxManage setextradata DEBIAN VBoxInternal2/SharedFoldersEnableSymlinksCreate/FOLDER_SHARE 1

4. Reboot your virtual machine.
5. Enjoy the stunning power of symlinks again*

* You ask why again ? Because this was possible by default prior to VirtualBox 4.1.8. It has been added for additional security.

Port Forwarding to a VirtualBox Debian

Set up port forwarding:

VBoxManage modifyvm "VM name" --natpf1 "to_ssh,tcp,,22,,22"
VBoxManage modifyvm "VM name" --natpf1 "to_http,tcp,,80,,80"
VBoxManage modifyvm "VM name" --natpf1 "to_https,tcp,,443,,443"

Replace “VM name” and <GUEST IP> with your desired configuration. If your guest OS use DHCP then can be omitted. natpf1 stands is the network interface to use. In my case NAT is Network Adapter 1.

To remove them again:

VBoxManage modifyvm "VM name" --natpf1 delete "to_ssh"

‘VBoxManage’ should be installed on your host os by the VirtualBox installer.

PDF Sam command line on OS X

Today I struggled with the usage of PDFSam on my Mac.
I just needed the command line tool, not the gui. On Linux it was

apt-get install pdfsam

On my OS X I dowloaded the .dmg from www.pdfsam.org and created this tiny ‘alias’ in my ~/.profile.

alias pdfsam='java -Dlog4j.configuration=console-log4j.xml \ 
-classpath /Applications/pdfsam-2.2.1.app/Contents/Resources\ 
/Java/pdfsam-2.2.1.jar org.pdfsam.console.ConsoleClient "$@"'

On command line I can now use it like I use it on Linux.

Netbeans 7 and the missing JAX-RPC plugin

I haven’t used Netbeans JAX binding for over a year.
And the last time I used it, it was easy to find in the plugin menu of Netbeans 6.
Now I’ve upgraded to Netbeans 7 and wasn’t able to find the plugin.
Happily I found a page that advised me to add

http://deadlock.netbeans.org/hudson/job/nbms-and-javadoc/lastStableBuild/artifact/nbbuild/nbms/updates.xml.gz

to the Update Center.
After that I got a huge list of updates, and my missing JAX-RPC.
Now just: install, restart, update-again.
Et voila, there is the missing “JAXB Binding” menu point that was searching for.

test proxy speed with bash and wget

I need to test the speed of some proxy server. Here’s a little script on how I achieved this.
I have a text-file ‘proxy.list’ which looks like: (I took out the last two digits of the ip).

...[lot's of ip's]...
193.196.*.*:3124       Germany
143.205.*.*:3127      Austria
64.161.*.*:3128         United States
.....

Here is the script which will run through the list of all proxy and will download 5 test pages from a specific site. Then it will determine the period of time which is needed for execution. It’s create/append to a file ‘time.list’ which will contain the needed Information to determine the best proxies. You also need to create a subdirectory called ‘raw_proxy’ where the raw html code is save that you retrieve from the proxies. The files are named ”raw_proxy/$ip.$port.$i.tmp’ where $i is the i.th test page you downloaded. I need to keep those files to determine if the proxy send me the right file or e.g. a login page .

#!/bin/bash
size=$(cat proxy.list | wc -l)
while read proxy
do
    #determine the first parameter (IP:Port)
    ad=$(echo $proxy | awk '{print $1}')
    ip=${ad%:*}   #extract ip
    port=${ad#*:} #extract port
    #set and export the proxy settings
    http_proxy=$ip:$port && HTTP_PROXY=$http_proxy && export http_proxy HTTP_PROXY
    #save start timestamp
    start=$(date +%s)
    #download 5 pages #(yes I know 'seq' but I'm on Mac and I needed a sth. quick&dirty)
    for i in $(echo "1 2 3 4 5")
    do
        #use wget to retrive the page. We want to try 1 time and set some specific timeouts. + we force to use a Mozilla User agent to hide that we are using wget.
    	wget -O "raw_proxy/$ip.$port.$i.tmp" --tries=1 --dns-timeout=10 --connect-timeout=8 --read-timeout=15 -U "Mozilla/5.0 (compatible; Konqueror/3.2; Linux)" "http://www.yourTestPage.com/$i.txt" &> /dev/null
    done
    #save end timestamp
    end=$(date +%s)
    #calculate the difference
    diff=$(( end - start ))
    #append this info to time.list
    echo -e "$ip:$port\t$diff" >> time.list
    #to have a nice and shiny output I use figlet, this is optional, if you don't want it comment out next 3 lines or just remove ' | figlet'
    clear
    echo "PC: #"$size" - "$diff"s" | figlet
    sleep 1
    size=$(( size-1 ))
done < proxy.list

If you used figlet your output looks like this:

                                                           
 ____   ____       _  _    __  _____ _           ____   ___      
|  _ \ / ___|_   _| || |_ / /_|___ // |         |___ \ / _ \ ___ 
| |_) | |   (_) |_  ..  _| '_ \ |_ \| |  _____    __) | | | / __|
|  __/| |___ _  |_      _| (_) |__) | | |_____|  / __/| |_| \__ \
|_|    \____(_)   |_||_|  \___/____/|_|         |_____|\___/|___/

It shows how much proxies need to be checked and shows the last execution time.

After the script has finished you need to get a list of which proxy was best.
This is the command line which evaluates everything and gives me back a list of ip's sorted by access time. It also removes all proxies where the downloaded page had a size of 0B.

#command line to list proxy with lowest time to download
clear && while read line; do ip=${line% *};time=$(echo $line | awk '{print $2}');ip=${ip%:*};echo -e $ip"\t"$time"\t"$(ls -alshr raw_proxy/ | grep 1.tmp | grep $ip | awk '{print $6}'); done < <(tr -s ' ' < time.list | sort -n -r -k2 | cut -d' ' -f3) | grep -v "0B"

This is the output:

201.22.*.*	43	52K
196.213.*.*	43	13K
....
147.102.*.*	1	2,1K
132.227.*.*	1	2,1K
....
130.75.*.*	1	52K

If you know the filesize the you can append a

 | grep "52K"

to the last command to show only files which have the right size.
This is it 😉

I know that out there are better and fast implementations to do this but ...
but it was fun

OS X – AppleScript to open Apps Hidden

I need to open Safari with a specific URL on command line. The problem is that this has to happen completly in background and should be started through an existing bashscript.
Here is my solutiuon, it should apply to any other app in OS X SnowLeopard. You need to call the bash schript like this:


./open_safari_with_url.sh "http://www.site_i_want_to_visit.com"

“open_safari_with_url.sh” looks like this:


#!/bin/bash
osascript -e "
tell application \"Safari\" 
	tell application \"Finder\"
		set visible of process \"Safari\" to false
	end tell
	launch
	set the URL of document 1 to \"$1\"
end tell"

[EDIT] It seems that ‘launch’ in AppleScripts starts the app in background. So we don’t need to tell Finder to hide Safari anymore.

tell application "Safari" 
	launch
	set the URL of document 1 to "http://whatever..."
end tell"

open terminal here

Marc Liyanage wrote a nice Apple Script which easily allows you to switch from the Finder to the Terminal.
Just download it here and drag the script to the upper free space in the Finder.

Clicking it results in a new Terminal window which location is the same as the Finder location.
new Finder icon

Here’s how it looks.

P.S. this works under OS X 10.6 😉