Skip to content →

Tag: Computer Networking

Configure Selenium and Chrome to use Tor proxy

I’ve been trying to configure Selenium and Chrome to use Tor as proxy and constantly getting error messages like the following:

WebDriverException: Message: invalid argument: cannot parse capability: proxy
from invalid argument: Specifying ‘socksProxy’ requires an integer for ‘socksVersion’
(Driver info: chromedriver=2.44.609545 (c2f88692e98ce7233d2df7c724465ecacfe74df5),platform=Mac OS X 10.14.0 x86_64)

In the end I have to use HTTP proxy instead of SOCKS.

Install and start Tor:

brew install tor
brew services start tor

Install privoxy:

brew install privoxy

Configure privoxy (vi /usr/local/etc/privoxy/config) to chain it with Tor:

forward-socks5t   /               127.0.0.1:9050 .

Start privoxy by default on port 8118:

brew services start privoxy

Check if your traffic is proxied:

from selenium.webdriver.common.proxy import Proxy, ProxyType
from selenium import webdriver

prox = Proxy()
prox.proxy_type = ProxyType.MANUAL
prox.http_proxy = "http://localhost:8118"
prox.ssl_proxy = "http://localhost:8118"

capabilities = webdriver.DesiredCapabilities.CHROME
prox.add_to_capabilities(capabilities)


options = webdriver.ChromeOptions()
options.binary_location = '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'
# options.add_argument('headless')
# set the window size
options.add_argument('window-size=1881x1280')

# initialize the driver
driver = webdriver.Chrome(options=options, desired_capabilities=capabilities)

driver.get('http://httpbin.org/ip')

Now you should see a different IP than your real one.

Leave a Comment

Raspberry Pi as a toy web server

1. Tell my router to route certain traffic to the Raspberry Pi. For example, I route HTTP and SSH traffic to one of my Pi’s. I disabled password login for  SSH, using public key authentication instead: in /etc/ssh/sshd_config, use the following setting and restart SSH service using `sudo service ssh restart’.

RSAAuthentication yes
PubkeyAuthentication yes

# To disable password authentication:
ChallengeResponseAuthentication no
PasswordAuthentication no
UsePAM no

# To disable root login:
PermitRootLogin no

2. Getting my IP address of the Raspberry Pi. I firstly created a PHP script on my domain to record the IP address in a text file.

<?php                                                                            
                                                                                 
$token = 'secret';                   
                                                                                 
if ($_GET['token'] == $token) {                                                  
    if (!empty($_SERVER['HTTP_CLIENT_IP'])) {                                    
        $ip = $_SERVER['HTTP_CLIENT_IP'];                                        
    } elseif (!empty($_SERVER['HTTP_X_FORWARDED_FOR'])) {                        
        $ip = $_SERVER['HTTP_X_FORWARDED_FOR'];                                  
    } else {                                                                     
        $ip = $_SERVER['REMOTE_ADDR'];                                           
    }                                                                            
    $fp = fopen('home_ip.txt', 'w');                                             
    fwrite($fp, $ip);                                                            
}                                                                                
                                                                                 
?>

Then tell the Raspberry Pi to report its IP address every 5 minutes, using crontab:

pi@alice ~ $ crontab -l
*/5 * * * * curl daoyuan.li/home_ip.php?token=secret

After a while the IP address is recorded in the text file and updated every 5 minutes.

3. Optionally create a DNS record for the Pi. I use Cloudflare to manage DNS settings by myself, so just add/update an entry in Cloudflare’s settings. I point pi.daoyuan.li to the IP address of one Pi. This can be done automatically in the future.

4. Install Flask on the Pi.

sudo apt-get update
sudo apt-get install python-pip
sudo pip install Flask

5. Install nginx and uwsgi on the Pi.

sudo apt-get install nginx
sudo apt-get install build-essential python python-dev
sudo apt-get install python-virtualenv
sudo pip install uwsgi

6. Set up nginx along with uwsgi and Flask.

mkdir flask && cd $_
virtualenv env
. env/bin/activate
pip install Flask

Edit nginx config:

~/flask $ cat flask_nginx.conf 
server {
    listen      5000;
    server_name localhost;
    charset     utf-8;
    client_max_body_size 75M;

    location / { try_files $uri @flask; }
    location @flask{
        include uwsgi_params;
        uwsgi_pass unix:/home/pi/flask/uwsgi.sock;
    }
}
sudo rm /etc/nginx/sites-enabled/default
sudo ln -s flask_nginx.conf /etc/nginx/sites-enabled/default 
sudo service nginx restart

Edit uwsgi config:

~/flask $ cat flask_uwsgi.ini 
[uwsgi]
#application's base folder
base = /home/pi/flask

#python module to import
app = hello
module = %(app)

home = %(base)/env
pythonpath = %(base)

#socket file's location
socket = /home/pi/flask/uwsgi.sock

#permissions for the socket file
chmod-socket    = 666

#the variable that holds a flask application inside the module imported at line #6
callable = app

#location of log files
logto = /home/pi/flask/uwsgi.log

Create a simple Flask app:

~/flask $ cat hello.py
from flask import Flask
app = Flask(__name__)

@app.route("/")
def hello():
    return "Hello World!"

if __name__ == "__main__":
    app.run()

Start up uwsgi:

uwsgi --ini flask_uwsgi.ini &

7. Done! http://pi.daoyuan.li:5000/

 Update on June 24, 2014:

Getting the external address in step 2 can be done by running this command in Raspberry Pi:

curl ifconfig.me

See: http://www.commandlinefu.com/commands/view/5427/get-your-external-ip-address

Leave a Comment

mod_wsgi and mod_xsendfile on OS X 10.9 Mavericks

Updated on Nov 4, 2013: The following tricks may still work, however I have found a much easier solution. Simply install Xcode command line developer tools and you should be able to compile source code without issues:

$ xcode-select --install

After upgrading my Mac from 10.8 to 10.9 Mavericks my apache stopped working, so I have to reinstall mod_wsgi and mod_xsendfile. However, tricks are needed to compile and install these mods successfully.

Upgrade Xcode

Upgrade Xcode in App Store. As mentioned by Valerie:

I had to manually upgrade Xcode (after Mavericks upgrade) from the App Store & agree to its license because ./configure hung forever until I did that.

mod_wsgi

For mod_wsgi installation, create a soft link to OSX10.9.xctoolchain:

cd /Applications/Xcode.app/Contents/Developer/Toolchains/
sudo ln -s XcodeDefault.xctoolchain OSX10.9.xctoolchain

Then run configure under mod_wsgi source code directory:

mod_wsgi-3.4$ ./configure

It will generate a Makefile similar as follows:

#  Copyright 2007 GRAHAM DUMPLETON                                               
#                                                                                
#  Licensed under the Apache License, Version 2.0 (the "License");               
#  you may not use this file except in compliance with the License.              
#  You may obtain a copy of the License at                                       
#                                                                                
#      http://www.apache.org/licenses/LICENSE-2.0                                
#                                                                                
#  Unless required by applicable law or agreed to in writing, software           
#  distributed under the License is distributed on an "AS IS" BASIS,             
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.      
#  See the License for the specific language governing permissions and           
#  limitations under the License.                                                

APXS = /usr/sbin/apxs                                                            
PYTHON = /usr/local/bin/python                                                   

DESTDIR =                                                                        
LIBEXECDIR = /usr/libexec/apache2                                                

CPPFLAGS =  -I/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/include/python2.7 -DNDEBUG 
CFLAGS =   -Wc,'-arch x86_64'                                                    
LDFLAGS =  -Wl,-F/usr/local/Cellar/python/2.7.5/Frameworks -framework Python -u _PyMac_Error   -arch x86_64
LDLIBS =  -ldl  -framework CoreFoundation                                        

all : mod_wsgi.la                                                                

mod_wsgi.la : mod_wsgi.c                                                         
    $(APXS) -c $(CPPFLAGS) $(CFLAGS) mod_wsgi.c $(LDFLAGS) $(LDLIBS)             

$(DESTDIR)$(LIBEXECDIR) :                                                        
    mkdir -p $@                                                                  

install : all $(DESTDIR)$(LIBEXECDIR)                                            
    $(APXS) -i -S LIBEXECDIR=$(DESTDIR)$(LIBEXECDIR) -n 'mod_wsgi' mod_wsgi.la 

clean :                                                                          
    -rm -rf .libs                                                                
    -rm -f mod_wsgi.o mod_wsgi.la mod_wsgi.lo mod_wsgi.slo mod_wsgi.loT          
    -rm -f config.log config.status                                              
    -rm -rf autom4te.cache                                                       

distclean : clean                                                                
    -rm -f Makefile Makefile.in                                                  

realclean : distclean                                                            
    -rm -f configure

However, this Makefile is not correct and running ‘make’ the compiler will complain something like:

mod_wsgi.c:34:10: fatal error: 'httpd.h' file not found

Prepend the following line to CPPFLAGS value:

-I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/apr-1 -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/apache2

Save the Makefile and it will look something like:

#  Copyright 2007 GRAHAM DUMPLETON                                               
#                                                                                
#  Licensed under the Apache License, Version 2.0 (the "License");               
#  you may not use this file except in compliance with the License.              
#  You may obtain a copy of the License at                                       
#                                                                                
#      http://www.apache.org/licenses/LICENSE-2.0                                
#                                                                                
#  Unless required by applicable law or agreed to in writing, software           
#  distributed under the License is distributed on an "AS IS" BASIS,             
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.      
#  See the License for the specific language governing permissions and           
#  limitations under the License.                                                

APXS = /usr/sbin/apxs                                                            
PYTHON = /usr/local/bin/python                                                   

DESTDIR =                                                                        
LIBEXECDIR = /usr/libexec/apache2                                                

CPPFLAGS =  -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/apr-1 -I/Applications/Xcode. app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/apache2 -I/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/include/python2.7 -DNDEBUG
CFLAGS =   -Wc,'-arch x86_64'                                                             
LDFLAGS =  -Wl,-F/usr/local/Cellar/python/2.7.5/Frameworks -framework Python -u _PyMac_Error   -arch x86_64
LDLIBS =  -ldl  -framework CoreFoundation                                        

all : mod_wsgi.la                                                                

mod_wsgi.la : mod_wsgi.c                                                         
    $(APXS) -c $(CPPFLAGS) $(CFLAGS) mod_wsgi.c $(LDFLAGS) $(LDLIBS)             

$(DESTDIR)$(LIBEXECDIR) :                                                        
    mkdir -p $@                                                                  

install : all $(DESTDIR)$(LIBEXECDIR)                                            
    $(APXS) -i -S LIBEXECDIR=$(DESTDIR)$(LIBEXECDIR) -n 'mod_wsgi' mod_wsgi.la 

clean :                                                                          
    -rm -rf .libs                                                                
    -rm -f mod_wsgi.o mod_wsgi.la mod_wsgi.lo mod_wsgi.slo mod_wsgi.loT          
    -rm -f config.log config.status                                              
    -rm -rf autom4te.cache                                                       

distclean : clean                                                                
    -rm -f Makefile Makefile.in                                                  

realclean : distclean                                                            
    -rm -f configure

Then make && install:

mod_wsgi-3.4$ make
mod_wsgi-3.4$ sudo make install

Modify /etc/apache2/httpd.conf to enable mod_wsgi:

LoadModule wsgi_module libexec/apache2/mod_wsgi.so

mod_xsendfile

Use the following command to compile and install mod_xsendfile:

sudo apxs -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/apr-1 -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/apache2 -cia mod_xsendfile.c

Enabling HTTPS/SSL

After upgrading OS X the apache configuration was reset, but your original config is save to /etc/apache2/httpd.conf.pre-update. I need to enable SSL in httpd.conf again by uncommenting the following line:

# Include /private/etc/apache2/extra/httpd-ssl.conf

Restart Apache and everything should work fine

sudo apachectl restart
3 Comments

OpenVPN client TLS-Auth problem on Windows

It seems that the OpenVPN Client on windows does not support TLS-Auth with a separate key file. So instead, you can paste your key contents in your openvpn client’s config file and use some thing like the following (inline ta.key):

client
dev tun
proto udp

# enter the server's hostname
# or IP address here, and port number
remote server_ip port_number

resolv-retry infinite
nobind
persist-key
persist-tun

# Use the full filepaths to your
# certificates and keys
ca "C:\\yabroad.tblk\\ca.crt"
cert "C:\\yabroad.tblk\\client.crt"
key "C:\\yabroad.tblk\\client.key"

key-direction 1

<tls-auth>
#
# 2048 bit OpenVPN static key
#
-----BEGIN OpenVPN Static key V1-----
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
-----END OpenVPN Static key V1-----
</tls-auth>


ns-cert-type server
comp-lzo
verb 1

When not using the above but using something like:

tls-auth ta.key 1  # which works for tunnelblick

The server log shows something like:

Thu Apr 18 20:55:26 2013 TLS Error: incoming packet authentication failed from [AF_INET] IP_ADDRESS:60454
Thu Apr 18 20:55:57 2013 Authenticate/Decrypt packet error: packet HMAC authentication failed
20 Comments

Technical Details

We define a BPF filter as “icmp and dst host <bouncer_ip>”, which filters out all none-ICMP packets and packets not destined to the bouncer. The we compile this filter and set the filter to the capture device.

When the capture device captures a packet, a process_pkt function is called to process the packet. And that is where we validate the packets and then, if the packet is valid, update
the packet and then send it out, or write the packet to a dump file.

The process of validating the packets are as follows:

  1. Validate checksum of IP header.
  2. Validate TTL of IP.
  3. Validate IP source address
  4. Validate ICMP header checksum. 
  5. Validate ICMP type and code. 

Since we write the packet to a dump file in a separate function, so a pointer to the dump file handler is passed to the processing function when the process_pkt function is called. The same method is used to pass server IP address and test mode flag.

We keep a linked list of all the ICMP echo requests. When a ICMP echo reply is received, we go through the linked list to find out where
the original echo request comes from.

The code is as follows.

bouncer.h:

/* Global definitions for the port bouncer
 * Packet headers and so on
 */

#define _BSD_SOURCE 1

#include 
#include 
#include 
#include 
#include 

/* PCAP declarations*/
#include 

/* Standard networking declaration */
#include 
#include 
#include 

/*
 * The following system include files should provide you with the 
 * necessary declarations for Ethernet, IP, and TCP headers
 */

#include 
#include 
#include 
#include 

/* Add any otherdeclarations you may need here... */

#define MAX_PACKET_SIZE 65535

/* Ethernet addresses are 6 bytes */
//#define ETHER_ADDR_LEN	6

/* Ethernet header */
struct sniff_ethernet {
    u_char ether_dhost[ETHER_ADDR_LEN]; /* Destination host address */
    u_char ether_shost[ETHER_ADDR_LEN]; /* Source host address */
    u_short ether_type; /* IP? ARP? RARP? etc */
};

/* IP header */
struct sniff_ip {
    u_char ip_vhl; /*version << 4 | header length >> 2 */
    u_char ip_tos; /* type of service */
    u_short ip_len; /* total length */
    u_short ip_id; /* identification */
    u_short ip_off; /* fragment offset field */
#define IP_RF 0x8000		/* reserved fragment flag */
#define IP_DF 0x4000		/* dont fragment flag */
#define IP_MF 0x2000		/* more fragments flag */
#define IP_OFFMASK 0x1fff	/* mask for fragmenting bits */
    u_char ip_ttl; /*time to live */
    u_char ip_p; /* protocol */
    u_short ip_sum; /* checksum */
    struct in_addr ip_src, ip_dst; /* source and dest address */
};
#define IP_HL(ip)		(((ip)->ip_vhl) & 0x0f)
#define IP_V(ip)		(((ip)->ip_vhl) >> 4)

/* ICMP header */
struct sniff_icmp {
    u_char icmp_type;
#define ICMP_ECHO 0x8
#define ICMP_REPLY 0x0
    u_char icmp_code;
    u_int16_t icmp_sum;
    u_int16_t icmp_id;
u_int16_t icmp_sequence;
};

#define SIZE_ETHERNET 14
#define SIZE_ICMP 8

#define IP_QUAD(ip)  (ip)>>24,((ip)&0x00ff0000)>>16,((ip)&0x0000ff00)>>8,((ip)&0x000000ff)

struct request{
    struct request* next;
    struct sniff_ip* ip;
    struct sniff_icmp* icmp;
};

process_pkt.c:

#include "bouncer.h"

/* CRC
 * Adopted from http://www.netfor2.com/ipsum.htm
 */
typedef unsigned short u16;
typedef unsigned long u32;

u16 ip_sum_calc(u16 len_ip_header, u16 buff[]) {
    u16 word16;
    u32 sum =0;
    u16 i;

    // make 16 bit words out of every two adjacent 8 bit words in the packet
    // and add them up
    for (i = 0; i < len_ip_header; i = i + 2) {
        word16 = ((buff[i] << 8)&0xFF00)+(buff[i + 1]&0xFF);
        sum = sum + (u32) word16;
    }

    // take only 16 bits out of the 32 bit sum and add up the carries
    while (sum >> 16)
        sum = (sum & 0xFFFF)+(sum >> 16);

    // one'scomplement the result
    sum = ~sum;

    return ((u16) sum);
}

void process_pkt(u_char *args, struct pcap_pkthdr *header,
        u_char *packet) {
    if(header->caplen < header->len){
        fprintf(stderr, "ERROR: Packet data not captured completely\n");
        return;
    }
    /* Main function to perform the bouncing */
    u_char testmode = *args;
    char *serv_addr = (char *) (args + 1);
    u_int32_t dst_addr =inet_addr(serv_addr);
    //pcap_dumper_t *dumper = NULL;
    u_char *dumper = NULL;
    memcpy(&dumper, args + 2 + strlen(serv_addr), sizeof (pcap_dumper_t *));
    struct request ** prequests = NULL;
    memcpy(&prequests, args + 2 + strlen(serv_addr) + sizeof (pcap_dumper_t *), sizeof (struct request**));
    //struct request *requests = *prequests;

    /* Typecasting packet*/
    struct sniff_ethernet *ethernet; /* The ethernet header */
struct sniff_ip *ip; /* The IP header */
    struct sniff_icmp *icmp; /* The ICMP header */
    char *padding; /* Packet padding */
    u_int size_ip;
    ethernet = (struct sniff_ethernet*) (packet);
    ip = (struct sniff_ip*) (packet + SIZE_ETHERNET);
    size_ip = IP_HL(ip)*4;
    u_int32_t srcip = ntohl(ip->ip_src.s_addr);
    if (size_ip < 20 || size_ip > header->len - SIZE_ETHERNET) {
        fprintf(stderr, "ERROR: Invalid IP header length: %ubytes\n", size_ip);
        if (testmode == 1) {
            header->caplen = 0;
            header->len = 0;
            pcap_dump(dumper, header, packet);
        }
        return;
    }
    icmp = (struct sniff_icmp*) (packet + SIZE_ETHERNET + size_ip);
    padding = (u_char *) (packet + SIZE_ETHERNET + size_ip + SIZE_ICMP);

    /* Validate the packet */
    /* Validate IP header */
    /* Check IP version */
    u_char v = ip->ip_vhl>> 4;
    if(v != 4){
        fprintf(stderr, "ERROR: Wrong IP verison %d\n", v);
        return;
    }
    /* Validate CRC */
    u16 ipbuf[size_ip];
    u16 ipsum = ntohs(ip->ip_sum);
    int i;
    for (i = 0; i < size_ip; i++) {
        if (i == 10 || i == 11)
            ipbuf[i] = 0x00;
        else
            ipbuf[i] = *((u_char *) (packet + SIZE_ETHERNET + i));
    }
    if (ipsum != ip_sum_calc(size_ip, ipbuf)) {
fprintf(stderr, "ERROR: ip checksum mismatch. Dropping packet from %u.%u.%u.%u\n",
                IP_QUAD(srcip));
        if (testmode == 1) {
            header->caplen = 0;
            header->len = 0;
            pcap_dump(dumper, header, packet);
        }
        return;
    }
    /* Validate TTL */
    if (ip->ip_ttl <= 0) {
        fprintf(stderr, "ERROR: ip TTL expired. Dropping packet from %u.%u.%u.%u\n",
IP_QUAD(srcip));
        if (testmode == 1) {
            header->caplen = 0;
            header->len = 0;
            pcap_dump(dumper, header, packet);
        }
        return;
    }
    /* Validate IP source address */
    if (srcip == 0x00000000 || srcip >= 0xE0000000
            || (srcip & 0x000000FF) == 0x000000FF || (srcip & 0x000000FF) == 0x00000000) {
        fprintf(stderr, "ERROR: ip source address invalid. Dropping packetfrom %u.%u.%u.%u\n",
                IP_QUAD(srcip));
        if (testmode == 1) {
            header->caplen = 0;
            header->len = 0;
            pcap_dump(dumper, header, packet);
        }
        return;
    }

    /* Validate ICMP header */
    /* Validate ICMP type and code */
    if (!(icmp->icmp_code == 0 && (icmp->icmp_type == 0 || icmp->icmp_type == 8))) {
        fprintf(stderr, "ERROR: icmp type orcode unsupported. Dropping packet from %u.%u.%u.%u\n",
                IP_QUAD(srcip));
        if (testmode == 1) {
            header->caplen = 0;
            header->len = 0;
            pcap_dump(dumper, header, packet);
        }
        return;
    }
    /* Validate CRC */
    int plen = header->len - SIZE_ETHERNET - size_ip - 8;
    u16 icmpbuf[8 + plen];
    u16 icmpsum = ntohs(icmp->icmp_sum);
    for (i = 0; i < 8;i++) {
        if (i == 2 || i == 3)
            icmpbuf[i] = 0x00;
        else
            icmpbuf[i] = *((u_char *) (packet + SIZE_ETHERNET + size_ip + i));
    }
    for (i = 0; i < plen; i++) {
        icmpbuf[i + 8] = *((u_char *) (padding + i));
    }
    if (icmpsum != ip_sum_calc(8 + plen, icmpbuf)) {
        fprintf(stderr, "ERROR: icmp checksum mismatch. Dropping packet from %u.%u.%u.%u\n",
                IP_QUAD(srcip));
        if(testmode == 1) {
            header->caplen = 0;
            header->len = 0;
            pcap_dump(dumper, header, packet);
        }
        return;
    }

    /* Update the packet */
    if (icmp->icmp_type == 0) {
        /* ICMP echo reply */
        ip->ip_src.s_addr = ip->ip_dst.s_addr;
        /* Search the linked list for client address */
        if (*prequests == NULL) {
            fprintf(stderr, "ERROR: process_pkt: nullrequest linked list\n");
            return;
        } else {
            /* Find where the request comes from */
            struct request *r;
            for (r = *prequests; r != NULL; r = r->next) {
                if (r->icmp->icmp_id == icmp->icmp_id && r->icmp->icmp_sequence == icmp->icmp_sequence) {
                    ip->ip_dst.s_addr = r->ip->ip_src.s_addr;
                    break;
                }
}
            if (r == NULL) {
                fprintf(stderr, "ERROR: no match echo requests in stack\n");
                return;
            }
            /* Remove the request from the linked list */
            struct request *rr = *prequests;
            if (rr == r) {
                free(rr->icmp);
                free(rr->ip);
                *prequests = r->next;
                free(r);
            } else {
                while (rr!= r)
                    rr = rr->next;
                free(rr->icmp);
                free(rr->ip);
                rr->next = r->next;
                free(r);
            }
        }
    } else {
        /* ICMP echo request */
        /* Add new request to linked list */
        struct request *r;
        r = malloc(sizeof (struct request));
        if (r == NULL) {
            perror("ERROR: process_pkt:malloc");
            return;
        }
        struct sniff_ip *ipt = malloc(sizeof (struct sniff_ip));
        if (ipt == NULL) {
            perror("ERROR: process_pkt: malloc");
            return;
        }
        memcpy(ipt, ip, sizeof (struct sniff_ip));
        r->ip = ipt;
        struct sniff_icmp* icmpt = malloc(sizeof (struct sniff_icmp));
        if (icmpt == NULL) {
            perror("ERROR: process_pkt: malloc");
return;
        }
        memcpy(icmpt, icmp, sizeof (struct sniff_icmp));
        r->icmp = icmpt;
        r->next = *prequests;
        *prequests = r;
        /* Update destination address */
        ip->ip_src.s_addr = ip->ip_dst.s_addr;
        ip->ip_dst.s_addr = dst_addr;
    }
    /* Recaculate CRC */
    for (i = 0; i < size_ip; i++) {
        if (i == 10 || i == 11)
            ipbuf[i] = 0x00;
        else
ipbuf[i] = *((u_char *) (packet + SIZE_ETHERNET + i));
    }
    ip->ip_sum = htons(ip_sum_calc(size_ip, ipbuf));
    if (testmode == 0) {
        pcap_t *handle = NULL;
        char errbuf[PCAP_ERRBUF_SIZE], *device = "tap0";
        memset(errbuf, 0, PCAP_ERRBUF_SIZE);
        if ((handle = pcap_open_live(device, MAX_PACKET_SIZE, 1, 512, errbuf)) == NULL) {
            fprintf(stderr, "ERROR: %s\n", errbuf);
            exit(1);
        }
        /*Send the packet to network */
        if (pcap_sendpacket(handle, packet, header->len) != 0) {
            perror("ERROR: process_pkt: pcap_sendpacket");
            exit(1);
        }
        pcap_close(handle);
        fprintf(stderr, "Bouncer: packet sent\n");
    } else {
        /* Or put it back on stdout */
        int i = 0;
        for (i = 0; i < ETHER_ADDR_LEN; i++)
            ethernet->ether_shost[i] = 1;
        if(icmp->icmp_type == 0) {
            for (i = 0; i < ETHER_ADDR_LEN; i++)
                ethernet->ether_dhost[i] = 2;
        } else if (icmp->icmp_type == 8) {
            for (i = 0; i < ETHER_ADDR_LEN; i++)
                ethernet->ether_dhost[i] = 3;
        }
        pcap_dump(dumper, header, packet);

////////////////////////////////////////////////////////////////////////
/*
        pcap_t *handle = NULL;
        char errbuf[PCAP_ERRBUF_SIZE], *device = "tap0";
        memset(errbuf, 0, PCAP_ERRBUF_SIZE);
        if ((handle = pcap_open_live(device, MAX_PACKETS_NO, 1, 512, errbuf)) == NULL) {
            fprintf(stderr, "ERROR: %s\n", errbuf);
            exit(1);
        }
        if (pcap_sendpacket(handle, packet, header->len) != 0) {
perror("ERROR: process_pkt: pcap_sendpacket");
            exit(1);
        }
        pcap_close(handle);
        fprintf(stderr, "Packet sent\n");
*/
        ////////////////////////////////////////////////////////////////////////

    }
    return;
}
Leave a Comment

Multiple Sessions

A linked list of all RUDP sockets is maintained. When rudp_socket() is called, an RUDP socket is created and added to the linked list. An RUDP socket keeps a record of the pees/sessions it talks with. When RUDP receives a packet from an unknown socket address, or when RUDP receives a send packet request to an unknown socket address, a new session is created. And for each session, a linked list of all buffered packets is kept.

Leave a Comment

Session Establishment and Tearing Down

When rudp_sendto() is called, the protocol first check if there exists a session between the sender and receiver. If not, the protocol will try to setup a session by sending RUDP_SYN messages. And the packet the application wants to send will be buffered in the created session. After an RUDP_ACK message is received, the server side socket start sending out packets. Go back N protocol is used to control the sending process. After the protocol receives a rudp_close() signal, it will first check whether there are still active sessions and packets in the sending buffer. If not, the protocol will send out RUDP_FIN messages and after receiving RUDP_ACKs, the session is torn down.

Leave a Comment

RUDP Overview

RUDP is a protocol that ensures transfer reliability with UDP. A sliding window protocol (Go back N) is used to realize reliability. Using RUDP, applications can send and receive data packets without worrying about lost packets.

The lines in red signifies state change for RUDP clients (receiver side); while the black lines signifies state change for RUDP servers (sender side).

Leave a Comment

Transferring Voice Data

This is done using JMF (Java Media Framework). Voice data is read from a file using `MediaLocator’. It is used as the data source for a `Processor’. The `Processor’ specifies and converts the audio data to a certain format and then output the converted data into a `DataSink’. The `DataSink’ then transfers the stream to its destination address and port.

Leave a Comment

Multiple Sessions

A SessionManager' is used to manage all the sessions. The SessionManager’ keeps a list of sessions. When it receives data, it will first check which session the data belongs to. Then the data is given to the corresponding session. If it does not belong to any session in the list, a new session will be created and added to the session list.

The basic code looks like this:


import java.io.IOException;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
import java.net.SocketException;
import java.util.ArrayList;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
 *
 * @author Daoyuan
 */
public class SessionManager extends Thread {

String user;
    String host;
    int port;
    String voiceFile;
    String defaultVoice;
    boolean listening = true;
    public static ArrayList sessionList;
    DatagramSocket serverSocket = null;

    public SessionManager(String user, String host, int port, String voiceFile,
            String defaultVoice) {
        this.user = user;
        this.host = host;
        this.port = port;
        this.voiceFile = voiceFile;
this.defaultVoice = defaultVoice;
        sessionList = new ArrayList();
        try {
            serverSocket = new DatagramSocket(port);
        } catch (SocketException ex) {
            Logger.getLogger(SessionManager.class.getName()).log(Level.SEVERE, null, ex);
            System.out.println("Unable to listen on port " + port);
        }
    }

    @Override
    public void run() {

        byte[] receiveData = newbyte[2048];

        new Thread() {

            @Override
            public void run() {
                while (true) {
                    try {
                        Thread.sleep(5);
                    } catch (InterruptedException ex) {
                        Logger.getLogger(SessionManager.class.getName()).log(Level.SEVERE, null, ex);
                    }
                    if (sessionList.size() != 0) {
                        for (int i= 0; i < sessionList.size(); i++) {
                            if (sessionList.get(i).status.equals("destroyed")) {
                                sessionList.remove(i);
                            }
                        }
                    }
                }
            }
        }.start();

        while (listening) {
            DatagramPacket receivePacket = new DatagramPacket(receiveData, receiveData.length);
try {
                serverSocket.receive(receivePacket);
            } catch (IOException ex) {
                Logger.getLogger(SessionManager.class.getName()).log(Level.SEVERE, null, ex);
            }

            if (isNewSession(receivePacket)) {
                System.out.println("New session created!");
            }
        }

        serverSocket.close();
    }

    public void stopListening() {
        listening = false;
}

    private boolean isNewSession(DatagramPacket receivePacket) {
        InetAddress IPAddress = receivePacket.getAddress();
        int remotePort = receivePacket.getPort();
        String sessionName = IPAddress + ":" + remotePort;
        if (sessionList.size() == 0) {
            Session s = new Session(this.serverSocket, IPAddress, remotePort, voiceFile, defaultVoice, user);
            s.setName(sessionName);
s.requests.add(receivePacket);
            sessionList.add(s);
            s.start();
            return true;
        } else {
            for (int i = 0; i < sessionList.size(); i++) {
                if (sessionList.get(i).getName().equals(sessionName)) {
                    sessionList.get(i).requests.add(receivePacket);
                    return false;
                }
            }

            Session s = newSession(this.serverSocket, IPAddress, remotePort, voiceFile, defaultVoice, user);
            s.setName(sessionName);
            s.requests.add(receivePacket);
            sessionList.add(s);
            s.start();
            return true;
        }
    }
}
Leave a Comment

Session Establishment and Tearing Down

Each session has a ‘status’, it can be ‘new’, ‘establishing’, ‘cancelling’, ‘established’, ‘tearingdown’ and ‘destroyed’.

When a new session is created, its status is ‘new’.

When an ‘INVITE’ is received, it sends out an ‘OK’ message and change its status to ‘establishing’.

After receiving an ‘ACK’ message the status will be changed to ‘established’.

Then begins the transferring of voice data using RTP.

When the sending finishes the status will become ‘tearingdown’.

A ‘BYE’ message is also sent to the client.

The status becomes ‘destroyed’ after getting ‘OK’ from the client.

When a ‘CANCEL’ message is received, the status becomes ‘cancelling’.

Then it sends back ‘OK’ and ‘Request Terminated’ messages.

After received an ‘ACK’, the status becomes ‘destroyed’.

The thread is as follows:


@Override
public void run() {

    while (true) {
        if (requests.size() == 0) {
            try {
                Thread.sleep(10);
            } catch (InterruptedException ex) {
                Logger.getLogger(Session.class.getName()).log(Level.SEVERE, null, ex);
            }
        } else {
            final DatagramPacket packet = requests.get(0);
            final String request = new String(packet.getData());
            System.err.println(request);
            sipRequest = new SIPRequest(request);
            if (request.toUpperCase().startsWith("INVITE")) {
                System.out.println("Got INVITE");
                if (!status.equals("new")) {
                    requests.remove(0);
                    continue;
                }

                String requestLine = request.substring(6).trim();
                String uri = requestLine.substring(0, requestLine.indexOf(' '));
                //System.out.println(uri);
                String usr = "";
                if (uri.toUpperCase().startsWith("SIP:")) {
                    usr = uri.substring(4, uri.indexOf('@'));
                } else {
                    usr = uri.substring(0, uri.indexOf('@'));
                }
                if (!usr.equalsIgnoreCase(user)) {
                    sendUserNotFound(sipRequest);
                    status = "tearingdown";
                    System.out.println("Status: " + status);
                    continue;
                }

                sendTrying(sipRequest);
                sendRinging(sipRequest);
                final String rtpport = request.substring(request.indexOf("m=audio") + 8, request.indexOf("m=audio") + 13);
                final String sdp = getSDP(request);
                new Thread() {

                    @Override
                    public void run() {
                        File file = new File("wav/" + voiceFile);
                        if (file.exists()) {
                            locator = new MediaLocator("file:/" + file.getAbsolutePath());
                        } else {
                            System.out.println("Using default voice file!");
                            locator = new MediaLocator("file:/" + file.getAbsolutePath());
                        }
                        transmitter = new Transmitter(locator, packet.getAddress().toString().substring(1), rtpport);
                        transmitter.prepare();
                    }
                }.start();

                new Thread() {

                    @Override
                    public void run() {
                        try {
                            Thread.sleep(5000);
                        } catch (InterruptedException ex) {
                            Logger.getLogger(Session.class.getName()).log(Level.SEVERE, null, ex);
                        }
                        if (status.equals("establishing")) {
                            sendOK(sipRequest, sdp, rtpport);
                        } else {
                            try {
                                transmitter.stop();
                            } catch (NullPointerException ex) {
                                Logger.getLogger(Session.class.getName()).log(Level.SEVERE, null, ex);
                                System.out.println("Transmitter already stopped!");
                            }
                        }
                    }
                }.start();
                requests.remove(0);
                status = "establishing";
                System.out.println("Status: " + status);
            } else if (request.toUpperCase().startsWith("ACK")) {
                System.out.println("Got ACK");
                //sendOK(sipRequest);
                requests.remove(0);
                if (status.equals("establishing")) {
                    status = "established";
                    System.out.println("Status: " + status);
                    new Thread() {

                        @Override
                        public void run() {
                            transmitter.start();

                            long duration = new VoiceInfo(locator).getDuration();
                            try {
                                Thread.sleep(duration);
                            } catch (InterruptedException ex) {
                                Logger.getLogger(Session.class.getName()).log(Level.SEVERE, null, ex);
                            }

                            try {
                                transmitter.stop();
                            } catch (NullPointerException ex) {
                                Logger.getLogger(Session.class.getName()).log(Level.SEVERE, null, ex);
                                System.out.println("Transmitter already stopped!");
                            }
                            if (!status.equals("tearingdown") && !status.equals("destroyed")) {
                                sendBye(sipRequest);
                                status = "tearingdown";
                                System.out.println("Status: " + status);
                            }
                        }
                    }.start();
                } else if (status.equals("tearingdown")) {
                    status = "destroyed";
                    System.out.println("Status: " + status);
                    break;
                } else if (status.equals("cancelling")) {
                    status = "destroyed";
                    System.out.println("Status: " + status);
                    break;
                }
            } else if (request.toUpperCase().startsWith("BYE")) {
                System.out.println("Got BYE");
                sendOKWithoutSDP(sipRequest);
                requests.remove(0);
                try {
                    transmitter.stop();
                } catch (NullPointerException ex) {
                    Logger.getLogger(Session.class.getName()).log(Level.SEVERE, null, ex);
                    System.out.println("Transmitter already stopped!");
                }
                status = "destroyed";
                System.out.println("Status: " + status);
                break;
            } else if (request.toUpperCase().startsWith("SIP/2.0 200")) {
                requests.remove(0);
                System.out.println("Got OK");
                if (status.equals("tearingdown")) {
                    status = "destroyed";
                    System.out.println("Status: " + status);
                    break;
                }
            } else if (request.toUpperCase().startsWith("CANCEL")) {
                requests.remove(0);
                System.out.println("Got CANCEL");
                status = "cancelling";
                System.out.println("Status: " + status);
                sendOKWithoutSDP(sipRequest);
                sendTerminated(sipRequest);
            } else {
                requests.remove(0);
            }
        }
    }
}
Leave a Comment

Generating Voice File

When the web server receives a POST' message from the web page, it will first check whether the message is valid. If so, freeTTS‘ is used to generate a voice file and the file is saved in the `wav’ directory with pre-configured file name.

The voice generator looks as follows:

import com.sun.speech.freetts.Voice;
import com.sun.speech.freetts.VoiceManager;
import com.sun.speech.freetts.audio.SingleFileAudioPlayer;

/**
 *
 * @author Daoyuan
 */
public class VoiceGenerator {

    String message;

    public VoiceGenerator(String message){
        this.message = message;
    }

    public void generateVoiceFile(String filename){

Voice voice;
        VoiceManager vm = VoiceManager.getInstance();
        voice = vm.getVoice("kevin16");
        voice.allocate();

        String baseName = filename.substring(0, filename.toLowerCase().indexOf(".wav"));
        SingleFileAudioPlayer sfap = new SingleFileAudioPlayer("wav/" + baseName, javax.sound.sampled.AudioFileFormat.Type.WAVE);
        voice.setAudioPlayer(sfap);
        voice.speak(message);

        sfap.close();

voice.deallocate();
    }
}
Leave a Comment

Non-ASCII characters in Email subject

In this system, Email subjects are encoded in Q' scheme, a scheme similar to quoted printable’. The format is “=?charset?Q?encoded subject?=”, in our case, the charset is `ISO-8859-15′.

For example, the Email object in this system is as follows:


importjava.text.SimpleDateFormat;
import java.util.Calendar;

/**
 * Email object.
 *
 * @author Daoyuan Li
 */
public class Mail {

    /**
     * Sender's Email address
     */
    private String from;

    /**
     * Recipient's Email address
     */
    private String to;

    /**
     * Email subject
     */
    private String subject;

    /**
     * Email message body
     */
    private Stringmessage;

    /**
     * Constructor of Mail
     * @param from Sender's Email address.
     * @param to Recipient's Email address.
     * @param subject Email subject.
     * @param message Email message.
     */
    public Mail(String from, String to, String subject, String message) {
        this.from = from;
        this.to = to;
        this.subject = subject;
        this.message = message;
    }

    /**
     * Get sender'sEmail address.
     * @return Sender's Email address.
     */
    public String getFrom() {
        return this.from;
    }

    /**
     * Get Recipient's Email address.
     * @return Recipient's Email address.
     */
    public String getTo() {
        return this.to;
    }

    /**
     * Get Email subject.
     * @return Email subject.
     */
    public String getSubject() {
        return this.subject;
}

    /**
     * Get Email message.
     * @return Email message.
     */
    public String getMessage() {
        return this.message;
    }

    /**
     * Set sender's Email address.
     * @param from Sender's Email address.
     */
    public void setFrom(String from) {
        this.from = from;
    }

    /**
     * Set Recipient's Email address.
     * @param to Recipient's Email address.
     */
public void setTo(String to) {
        this.to = to;
    }

    /**
     * Set Email subject.
     * @param subject Email subject.
     */
    public void setSubject(String subject) {
        this.subject = subject;
    }

    /**
     * Set Email message.
     * @param message Email message.
     */
    public void setMessage(String message) {
        this.message = message;
    }

    /**
     * Encode Email into MIMEformat.
     * @return The MIME encoded Email.
     */
    public String encode() {
        String FORMAT = "EEE, dd MMM yyyy HH:mm:ss Z";
        //String FORMAT = "yyyy-MM-dd HH:mm:ss Z";
        Calendar cal = null;
        cal = Calendar.getInstance();
        SimpleDateFormat sdf = new SimpleDateFormat(FORMAT);
        //System.out.println(sdf.format(cal.getTime()));
        String s = "From: <" + this.getFrom() + ">\n"
                + "To:<" + this.getTo() + ">\n"
                //+ "Subject: " + this.getSubject() + "\n"
                + "Subject: =?ISO-8859-15?Q?"
                + new QuotedPrintableEncoder().encodeWithoutLineBreak(this.subject) + "?=\n"
                + "Date: " + sdf.format(cal.getTime()) + "\n"
                + "Message-ID: " + cal.getTimeInMillis() + "@ik2213.lab\n"
                + "MIME-Version: 1.0\n"
                + "Content-Type:text/plain; charset=ISO-8859-15\n"
                + "Content-Transfer-Encoding: quoted-printable\n";
        return s.concat(new QuotedPrintableEncoder().encode(this.message));
    }
}
Leave a Comment

MIME Encoding

Quoted printable characters are encoded in the format =XX', where XX’ stands for the hexadecimal value of the character.

The encoder looks as follows:

import java.io.UnsupportedEncodingException;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
 * It encodes data into quoted printable format.
 *
* @author Daoyuan Li
 */
public class QuotedPrintableEncoder {
    /**
     * Encodes data into quoted printable format.
     * @param s Data to be encoded.
     * @return The encoded data.
     */
    public String encode(String s) {
        byte[] b = null;
        try {
            b = s.getBytes("ISO-8859-1");
        } catch (UnsupportedEncodingException ex) {
            Logger.getLogger(Mail.class.getName()).log(Level.SEVERE, null, ex);
return "";
        }
        String code = "";
        int wc = 0;
        for (int i = 0; i < b.length; i++) {
            byte c = b[i];
            if(c == 13){
                code = code.concat("\n");
                wc = 0;
                continue;
            } else if(c == 10) {
                //do nothing
                continue;
            }
            code = code.concat("=" + Integer.toHexString(c &255).toUpperCase());
            wc += 3;
            if (wc >= 75) {
                code = code.concat("=\n");
                wc = 0;
            }
        }
        return code;
    }

    /**
     * Encodes data into quoted printable format, without soft line breaks.
     * @param s Data to be encoded.
     * @return The encoded data.
     */
    public String encodeWithoutLineBreak(String s) {
        byte[] b = null;
        try{
            b = s.getBytes("ISO-8859-1");
        } catch (UnsupportedEncodingException ex) {
            Logger.getLogger(Mail.class.getName()).log(Level.SEVERE, null, ex);
            return "";
        }
        String code = "";
        for (int i = 0; i < b.length; i++) {
            byte c = b[i];
            if(c == 13){
                code = code.concat("\n");
                continue;
            } else if (c == 10){
//do nothing
                continue;
            }
            code = code.concat("=" + Integer.toHexString(c & 255).toUpperCase());
        }
        return code;
    }
}
Leave a Comment

URL Decoding

The charset of the form page is ISO-8859-15'. HTTP will encode the form message in URLEncoding, that is, space is replaced with +’; non-ASCII characters are encoded inthe format %XX', where XX’ stands for the hexadecimal value of the character.

The URL decoder in this system looks as follows:

importjava.io.UnsupportedEncodingException;
import java.util.Vector;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
 * UrlDecoder decodes URL encoded information into quoted printable characters.
 * 
 * @author Daoyuan Li
 */
public class UrlDecoder {

    /**
     * Decode from URL encoding to quoted printable.
     * @param url URL encoded message to be decoded.
     * @return Decoded quoted printable string.
     * @throwsException If anything wrong happens, an Exception is thrown.
     */
    public String decode(String url) throws Exception {
        String decoded = "";
        Exception ex = new Exception();
        Vector buf = new Vector();
        for (int i = 0; i < url.length(); i++) {
            if (url.charAt(i) == '%') {
                if (i + 2 >= url.length()) {
                    throw ex;
                }

                int d = -1;
try {
                    d = Integer.parseInt(url.substring(i + 1, i + 3), 16);
                } catch (NumberFormatException e) {
                    throw ex;
                }
                if (d > 255 || d < 0) {
                    throw ex;
                }
                buf.add(new Byte((byte)d));
                i += 2;
            } else if (url.charAt(i) == '+') {
                buf.add(new Byte((byte)' '));
} else {
                buf.add(new Byte((byte) url.charAt(i)));
            }
        }
        try {
            byte[] dcd = new byte[buf.size()];
            for (int j = 0; j < dcd.length; j++) {
                dcd[j] = buf.elementAt(j);
            }
            decoded = new String(dcd, "ISO-8859-1");
        } catch (UnsupportedEncodingException e) {
            Logger.getLogger(FormValidator.class.getName()).log(Level.SEVERE, null,e);
            throw ex;
        }
        return decoded;
    }
}
Leave a Comment