Skip to content →

Tag: Web

Migrating/Installing Prestashop to Amazon AWS EC2

Published 2015/02/25 by Daoyuan Li

Recently I migrated a Prestashop website to AWS/EC2 and the process is quite smooth. Here’s how to do it.

1. Get your Website data: including Prestashop website files and a latest database dump. Remove all the files except index.php in /cache/smarty/compile and /cache/smarty/cache. (See here.)

2. Launch your EC2 instance. But before doing that, check first if a pre-configured or managed Prestashop is available in your preferred region (more info here). If not, choose the appropriate instance type and get it running. In my case I chose Ubuntu Linux.

3. Install the AMP (Apache, MySQL and PHP). On Ubuntu you can simply do it with a few simple commands:

sudo apt-get update
sudo apt-get upgrade
sudo tasksel install lamp-server
sudo apt-get install phpmyadmin

Make sure that you remember your password for MySQL root user and phpmyadmin user.

4. Copy your database dump and website files to your EC2 instance using scp, for instance:

scp -i you-key.pem files.tar.gz ubuntu@your-ec2-instance.compute.amazonaws.com

5. Create a MySQL user for Prestashop, you don’t want to use the root user for this purpose. You can do this using phpmyadmin or a command line. Also, create a database for Prestashop and grant the user you’ve created just now all privileges on this database.

6. Import your database from the dump:

mysql -u root -p ps < db_dump.sql

7. Extract your website files on your EC2 instance. In my case, inside /var/www/html/. If there is an index.html file inside this directory, remove it first. Make sure you set the correct user and group to all the website files:

sudo chown -R www-data:www-data *
sudo chmod -R 755 *

Note that if don’t set the correct permissions, you may get complaints from Apache, like the following:

PHP Fatal error:  Uncaught exception ‘SmartyException’ with message ‘unable to write file /var/www/html/cache/smarty/compile/46/d9/69/wrt54ecd6c1856a72.61943173’ in /var/www/html/tools/smarty/sysplugins/smarty_internal_write_file.php:44

Stack trace:

#0 /var/www/html/tools/smarty/sysplugins/smarty_internal_template.php(201): Smarty_Internal_Write_File::writeFile(‘/var/www/html/c…’, ‘<?php /* Smarty…’, Object(Smarty))

#1 /var/www/html/tools/smarty/sysplugins/smarty_internal_templatebase.php(155): Smarty_Internal_Template->compileTemplateSource()

#2 /var/www/html/classes/controller/AdminController.php(1936): Smarty_Internal_TemplateBase->fetch(‘controllers/mod…’)

#3 /var/www/html/classes/controller/AdminController.php(2337): AdminControllerCore->initModal()

#4 /var/www/html/classes/controller/Controller.php(163): AdminControllerCore->init()

#5 /var/www/html/classes/Dispatcher.php(373): ControllerCore->run()

#6 /var/www/html/admin123/index.php(54): DispatcherCore->dispatch()

#7 {main}\n  thrown in /var/www/html/tools/smarty/sysplugins/smarty_internal_write_file.php on line 44

8. Enable mcrypt for PHP:

sudo updatedb
locate mcrypt.ini
> /etc/php5/mods-available/mcrypt.ini
locate mcrypt.so
> /usr/lib/libmcrypt.so.4
> /usr/lib/libmcrypt.so.4.4.8
> /usr/lib/php5/20121212/mcrypt.so
sudo vi /etc/php5/mods-available/mcrypt.ini
sudo php5enmod mcrypt
ls -al /etc/php5/cli/conf.d/20-mcrypt.ini
> lrwxrwxrwx 1 root root 31 Feb 24 19:32 /etc/php5/cli/conf.d/20-mcrypt.ini -> ../../mods-available/mcrypt.ini
ls -al /etc/php5/apache2/conf.d/20-mcrypt.ini
> lrwxrwxrwx 1 root root 31 Feb 24 19:32 /etc/php5/apache2/conf.d/20-mcrypt.ini -> ../../mods-available/mcrypt.ini
service apache2 restart

9. Update your Prestashop settings in the config/settings.inc.php file. Set the correct database information.

10. Enable mod_rewrite:

sudo a2enmod rewrite
sudo service apache2 restart

Also, make sure to add something like the following in your apache site configuration file:

        <Directory />
                Options FollowSymLinks
                AllowOverride all
        </Directory>
        <Directory /var/www/html/>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride all
                Order allow,deny
                allow from all
        </Directory>

And then remember to restart your apache server.

11. Install sendmail:

sudo apt-get install sendmail

12. Go to your Prestashop back office and reconfigure your domain. Go to Preferences -> SEO & URLs and scroll down to Set shop URL section.

Now everything should be OK, your new Prestashop website is available on your new domain!

If you have any issues, just drop a line here or email me.

4 Comments

Raspberry Pi as a toy web server

Published 2014/04/24 by Daoyuan Li

1. Tell my router to route certain traffic to the Raspberry Pi. For example, I route HTTP and SSH traffic to one of my Pi’s. I disabled password login for  SSH, using public key authentication instead: in /etc/ssh/sshd_config, use the following setting and restart SSH service using `sudo service ssh restart’.

RSAAuthentication yes
PubkeyAuthentication yes

# To disable password authentication:
ChallengeResponseAuthentication no
PasswordAuthentication no
UsePAM no

# To disable root login:
PermitRootLogin no

2. Getting my IP address of the Raspberry Pi. I firstly created a PHP script on my domain to record the IP address in a text file.

<?php                                                                            
                                                                                 
$token = 'secret';                   
                                                                                 
if ($_GET['token'] == $token) {                                                  
    if (!empty($_SERVER['HTTP_CLIENT_IP'])) {                                    
        $ip = $_SERVER['HTTP_CLIENT_IP'];                                        
    } elseif (!empty($_SERVER['HTTP_X_FORWARDED_FOR'])) {                        
        $ip = $_SERVER['HTTP_X_FORWARDED_FOR'];                                  
    } else {                                                                     
        $ip = $_SERVER['REMOTE_ADDR'];                                           
    }                                                                            
    $fp = fopen('home_ip.txt', 'w');                                             
    fwrite($fp, $ip);                                                            
}                                                                                
                                                                                 
?>

Then tell the Raspberry Pi to report its IP address every 5 minutes, using crontab:

pi@alice ~ $ crontab -l
*/5 * * * * curl daoyuan.li/home_ip.php?token=secret

After a while the IP address is recorded in the text file and updated every 5 minutes.

3. Optionally create a DNS record for the Pi. I use Cloudflare to manage DNS settings by myself, so just add/update an entry in Cloudflare’s settings. I point pi.daoyuan.li to the IP address of one Pi. This can be done automatically in the future.

4. Install Flask on the Pi.

sudo apt-get update
sudo apt-get install python-pip
sudo pip install Flask

5. Install nginx and uwsgi on the Pi.

sudo apt-get install nginx
sudo apt-get install build-essential python python-dev
sudo apt-get install python-virtualenv
sudo pip install uwsgi

6. Set up nginx along with uwsgi and Flask.

mkdir flask && cd $_
virtualenv env
. env/bin/activate
pip install Flask

Edit nginx config:

~/flask $ cat flask_nginx.conf 
server {
    listen      5000;
    server_name localhost;
    charset     utf-8;
    client_max_body_size 75M;

    location / { try_files $uri @flask; }
    location @flask{
        include uwsgi_params;
        uwsgi_pass unix:/home/pi/flask/uwsgi.sock;
    }
}
sudo rm /etc/nginx/sites-enabled/default
sudo ln -s flask_nginx.conf /etc/nginx/sites-enabled/default 
sudo service nginx restart

Edit uwsgi config:

~/flask $ cat flask_uwsgi.ini 
[uwsgi]
#application's base folder
base = /home/pi/flask

#python module to import
app = hello
module = %(app)

home = %(base)/env
pythonpath = %(base)

#socket file's location
socket = /home/pi/flask/uwsgi.sock

#permissions for the socket file
chmod-socket    = 666

#the variable that holds a flask application inside the module imported at line #6
callable = app

#location of log files
logto = /home/pi/flask/uwsgi.log

Create a simple Flask app:

~/flask $ cat hello.py
from flask import Flask
app = Flask(__name__)

@app.route("/")
def hello():
    return "Hello World!"

if __name__ == "__main__":
    app.run()

Start up uwsgi:

uwsgi --ini flask_uwsgi.ini &

7. Done! http://pi.daoyuan.li:5000/

 Update on June 24, 2014:

Getting the external address in step 2 can be done by running this command in Raspberry Pi:

curl ifconfig.me

See: http://www.commandlinefu.com/commands/view/5427/get-your-external-ip-address

Leave a Comment

Debugging Apache configuration files on Mac OS X

Published 2013/10/31 by Daoyuan Li

Sometimes I make some modification to my site’s apache config and apache stops functioning, but I cannot get enough information from /var/log/apache2/error.log. Here’s a command that will tell you where the problem is:

sudo /usr/sbin/httpd -k start -e Debug -E /dev/stdout

It starts apache for debugging and prints out the messages on screen, so that hopefully you can find something useful.

Leave a Comment

mod_wsgi and mod_xsendfile on OS X 10.9 Mavericks

Published 2013/10/27 by Daoyuan Li

Updated on Nov 4, 2013: The following tricks may still work, however I have found a much easier solution. Simply install Xcode command line developer tools and you should be able to compile source code without issues:

$ xcode-select --install

After upgrading my Mac from 10.8 to 10.9 Mavericks my apache stopped working, so I have to reinstall mod_wsgi and mod_xsendfile. However, tricks are needed to compile and install these mods successfully.

Upgrade Xcode

Upgrade Xcode in App Store. As mentioned by Valerie:

I had to manually upgrade Xcode (after Mavericks upgrade) from the App Store & agree to its license because ./configure hung forever until I did that.

mod_wsgi

For mod_wsgi installation, create a soft link to OSX10.9.xctoolchain:

cd /Applications/Xcode.app/Contents/Developer/Toolchains/
sudo ln -s XcodeDefault.xctoolchain OSX10.9.xctoolchain

Then run configure under mod_wsgi source code directory:

mod_wsgi-3.4$ ./configure

It will generate a Makefile similar as follows:

#  Copyright 2007 GRAHAM DUMPLETON                                               
#                                                                                
#  Licensed under the Apache License, Version 2.0 (the "License");               
#  you may not use this file except in compliance with the License.              
#  You may obtain a copy of the License at                                       
#                                                                                
#      http://www.apache.org/licenses/LICENSE-2.0                                
#                                                                                
#  Unless required by applicable law or agreed to in writing, software           
#  distributed under the License is distributed on an "AS IS" BASIS,             
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.      
#  See the License for the specific language governing permissions and           
#  limitations under the License.                                                

APXS = /usr/sbin/apxs                                                            
PYTHON = /usr/local/bin/python                                                   

DESTDIR =                                                                        
LIBEXECDIR = /usr/libexec/apache2                                                

CPPFLAGS =  -I/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/include/python2.7 -DNDEBUG 
CFLAGS =   -Wc,'-arch x86_64'                                                    
LDFLAGS =  -Wl,-F/usr/local/Cellar/python/2.7.5/Frameworks -framework Python -u _PyMac_Error   -arch x86_64
LDLIBS =  -ldl  -framework CoreFoundation                                        

all : mod_wsgi.la                                                                

mod_wsgi.la : mod_wsgi.c                                                         
    $(APXS) -c $(CPPFLAGS) $(CFLAGS) mod_wsgi.c $(LDFLAGS) $(LDLIBS)             

$(DESTDIR)$(LIBEXECDIR) :                                                        
    mkdir -p $@                                                                  

install : all $(DESTDIR)$(LIBEXECDIR)                                            
    $(APXS) -i -S LIBEXECDIR=$(DESTDIR)$(LIBEXECDIR) -n 'mod_wsgi' mod_wsgi.la 

clean :                                                                          
    -rm -rf .libs                                                                
    -rm -f mod_wsgi.o mod_wsgi.la mod_wsgi.lo mod_wsgi.slo mod_wsgi.loT          
    -rm -f config.log config.status                                              
    -rm -rf autom4te.cache                                                       

distclean : clean                                                                
    -rm -f Makefile Makefile.in                                                  

realclean : distclean                                                            
    -rm -f configure

However, this Makefile is not correct and running ‘make’ the compiler will complain something like:

mod_wsgi.c:34:10: fatal error: 'httpd.h' file not found

Prepend the following line to CPPFLAGS value:

-I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/apr-1 -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/apache2

Save the Makefile and it will look something like:

#  Copyright 2007 GRAHAM DUMPLETON                                               
#                                                                                
#  Licensed under the Apache License, Version 2.0 (the "License");               
#  you may not use this file except in compliance with the License.              
#  You may obtain a copy of the License at                                       
#                                                                                
#      http://www.apache.org/licenses/LICENSE-2.0                                
#                                                                                
#  Unless required by applicable law or agreed to in writing, software           
#  distributed under the License is distributed on an "AS IS" BASIS,             
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.      
#  See the License for the specific language governing permissions and           
#  limitations under the License.                                                

APXS = /usr/sbin/apxs                                                            
PYTHON = /usr/local/bin/python                                                   

DESTDIR =                                                                        
LIBEXECDIR = /usr/libexec/apache2                                                

CPPFLAGS =  -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/apr-1 -I/Applications/Xcode. app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/apache2 -I/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/include/python2.7 -DNDEBUG
CFLAGS =   -Wc,'-arch x86_64'                                                             
LDFLAGS =  -Wl,-F/usr/local/Cellar/python/2.7.5/Frameworks -framework Python -u _PyMac_Error   -arch x86_64
LDLIBS =  -ldl  -framework CoreFoundation                                        

all : mod_wsgi.la                                                                

mod_wsgi.la : mod_wsgi.c                                                         
    $(APXS) -c $(CPPFLAGS) $(CFLAGS) mod_wsgi.c $(LDFLAGS) $(LDLIBS)             

$(DESTDIR)$(LIBEXECDIR) :                                                        
    mkdir -p $@                                                                  

install : all $(DESTDIR)$(LIBEXECDIR)                                            
    $(APXS) -i -S LIBEXECDIR=$(DESTDIR)$(LIBEXECDIR) -n 'mod_wsgi' mod_wsgi.la 

clean :                                                                          
    -rm -rf .libs                                                                
    -rm -f mod_wsgi.o mod_wsgi.la mod_wsgi.lo mod_wsgi.slo mod_wsgi.loT          
    -rm -f config.log config.status                                              
    -rm -rf autom4te.cache                                                       

distclean : clean                                                                
    -rm -f Makefile Makefile.in                                                  

realclean : distclean                                                            
    -rm -f configure

Then make && install:

mod_wsgi-3.4$ make
mod_wsgi-3.4$ sudo make install

Modify /etc/apache2/httpd.conf to enable mod_wsgi:

LoadModule wsgi_module libexec/apache2/mod_wsgi.so

mod_xsendfile

Use the following command to compile and install mod_xsendfile:

sudo apxs -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/apr-1 -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/apache2 -cia mod_xsendfile.c

Enabling HTTPS/SSL

After upgrading OS X the apache configuration was reset, but your original config is save to /etc/apache2/httpd.conf.pre-update. I need to enable SSL in httpd.conf again by uncommenting the following line:

# Include /private/etc/apache2/extra/httpd-ssl.conf

Restart Apache and everything should work fine

sudo apachectl restart
3 Comments

Tastypie error handling with Unauthorized Exception

Published 2013/07/03 by Daoyuan Li

In Tastypie v0.9.15 unauthorized exception messages are not included in the response. The cause lies in the following code: (tastypie/resources.py, Line 601-602 in v0.9.15)

    def unauthorized_result(self, exception):
        raise ImmediateHttpResponse(response=http.HttpUnauthorized())

In the above snippet an empty response is returned, without including messages in `exception`.

In order to fix this, you can override this method in your model resources, for example:

    def unauthorized_result(self, exception):
        """Override tastypie method to return a reasonable error response."""
        raise ImmediateHttpResponse(
            response=HttpUnauthorized(
                json.dumps({'error': exception.message})
            )
        )

Hi, the company I’m working for (yabroad.com) is hiring Website Backend and Frontend Developers to our platform team. We are building an open platform for youngsters to travel beyond boarders and we offer youngsters internship, language study, travel and volunteer opportunities. Please contact me if you are interested.

Leave a Comment

Upgrading Tastypie from v0.9.11 to v0.9.12 and above

Published 2013/05/27 by Daoyuan Li

Here are my notes on how to upgrade Tastypie from version 0.9.11 to 0.9.12, as there’re no release notes for v0.9.12. Release notes for 0.9.13 and above are available here: http://django-tastypie.readthedocs.org/en/latest/release_notes/index.html.

1. override_urls() becomes prepend_urls(), the new name makes more sense since what the function does is to insert customized urls instead of replacing the default urls with your customized urls.

2. obj_* methods accepts different parameters. Before:

def obj_get(self, request=None, **kwargs):
    pass

Now obj_get, obj_create, etc. accepts bundle as the parameter apart from keyword arguments. You can get request object from the bundle:

def obj_get(self, bundle, **kwargs):
    request = bundle.request
    pass

3. apply_authorization_limits is no longer in use and it’s replace with a finer grained authorization mechanism. For example:

from tastypie.authorization import Authorization
from tastypie.exceptions import Unauthorized

class ProgramAuthorization(Authorization):
    def read_list(self, object_list, bundle):
        user = bundle.request.user
        if user.is_authenticated() and user.is_staff:
            return object_list
        else:
            return object_list.filter(is_published=True)

    def read_detail(self, object_list, bundle):
        if bundle.request.user.is_staff or bundle.obj.is_published:
            return True
        raise Unauthorized()

    def create_list(self, object_list, bundle):
        raise Unauthorized()

    def create_detail(self, object_list, bundle):
        if bundle.request.user.is_staff:
            return True
        raise Unauthorized()

    def update_list(self, object_list, bundle):
        raise Unauthorized()

    def update_detail(self, object_list, bundle):
        if bundle.request.user.is_staff:
            return True
        raise Unauthorized()

    def delete_list(self, object_list, bundle):
        raise Unauthorized()

    def delete_detail(self, object_list, bundle):
        raise Unauthorized()

Please note that *_list methods should return a list or raise exceptions; whereas *_detail methods should return boolean values or raise exceptions. For more information, see: http://django-tastypie.readthedocs.org/en/v0.9.12/authorization.html

Hi, the company I’m working for (yabroad.com) is hiring Website Backend and Frontend Developers to our platform team. We are building an open platform for youngsters to travel beyond boarders and we offer youngsters internship, language study, travel and volunteer opportunities. Please contact me if you are interested.

Leave a Comment

Installing mod_xsendfile on OS X Lion

Published 2012/11/14 by Daoyuan Li

First download mod_xsendfile.c from https://tn123.org/mod_xsendfile/

Then compile and install it:

sudo apxs -cia mod_xsendfile.c

Add the following line to /etc/apache2/httpd.conf at the end of the first block of “Load” statements:

LoadModule xsendfile_module libexec/apache2/mod_xsendfile.so

At last, restart the apache server

sudo apachectl restart

Hi, the company I’m working for (yabroad.com) is hiring Website Backend and Frontend Developers to our platform team. We are building an open platform for youngsters to travel beyond boarders and we offer youngsters internship, language study, travel and volunteer opportunities. Please contact me if you are interested.

Leave a Comment