Skip to content →

Category: Blog

Notes to myself: RTX 2070, Cuda, cudnn, caffe, and faceswap

Published 2019/05/19 by Daoyuan Li

Install NVIDIA driver for RTX 2070: https://www.geforce.com/drivers

Install CUDA 10.0: https://developer.nvidia.com/cuda-downloads

DO NOT re-install the drivers suggested by the CUDA installer:
Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 410.48?
(y)es/(n)o/(q)uit: n

CuDNN:

cuDNN Runtime Library for Ubuntu18.04 (Deb)

cuDNN Developer Library for Ubuntu18.04 (Deb)

Caffe:

CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
CUDA_cublas_device_LIBRARY (ADVANCED)
linked by target “caffe” in directory /home/dli/Projects/caffe/src/caffe

Upgrade cmake:

face_swap

Leave a Comment

Configure Selenium and Chrome to use Tor proxy

Published 2018/11/29 by Daoyuan Li

I’ve been trying to configure Selenium and Chrome to use Tor as proxy and constantly getting error messages like the following:

WebDriverException: Message: invalid argument: cannot parse capability: proxy
from invalid argument: Specifying ‘socksProxy’ requires an integer for ‘socksVersion’
(Driver info: chromedriver=2.44.609545 (c2f88692e98ce7233d2df7c724465ecacfe74df5),platform=Mac OS X 10.14.0 x86_64)

In the end I have to use HTTP proxy instead of SOCKS.

Install and start Tor:

Install privoxy:

Configure privoxy (vi /usr/local/etc/privoxy/config) to chain it with Tor:

Start privoxy by default on port 8118:

Check if your traffic is proxied:

Now you should see a different IP than your real one.

Leave a Comment

Imposter Syndrome

Published 2018/10/09 by Daoyuan Li

Internet is a good thing: it allows us to see and experience things that were not possible decades ago. On the other hand, it’s sometimes also overwhelming. When I see capable people doing amazing stuff on the Internet, very often I feel I’m not worth of what people think I am. Occasionally I thought it’s because I’m shy and I’m usually not comfortable fully expressing my thoughts. In other occasions I’m simply not patiently enough to argue otherwise.

But it’s not just me. A lot of people are experiencing the same feeling and I’m starting to realize that I may be suffering from the imposter syndrome. After all, after so many years my personality has changed a lot. I’m no longer shy. I’m much more confident than I was. And I’m also a more capable person so I deserve a better life. It doesn’t feel right to be satisfied with what I’m getting. I’m better than that!

Meeting with people confirms that. There are a lot of people pretending to be expert in something and it seems they are enjoying quite a lot, although I can spot right on that they’re not what they believe they are. They’re not ashamed, why should I feel bad for myself? I’m much better a person than those real imposters, of course I deserve better things!

It’s more than a year since I wrote a blog post last time. A lot has happened during the past year. After years of depressing research career, I’ve published in a good conference and successfully defended my PhD. I’ve started working (again) in the industry. I’ve also found a long term direction that should keep me busy in the next five years. We’ve also got our foothold in Luxembourg, although there remains a lot to complete, it’s a good starting point. I’ll try to blog more often about what’s happening in my life, just to keep track of my thoughts and experiences, so that I can look back to them later and say “boy, I didn’t know I was so naïve a while ago.”

I’ve also updated my blog theme and fixed the HTTPS issue with WordPress and Cloudflare: I only needed to install the CloudFlare Flexible SSL plugin and enable “Automatic HTTPS Rewrites” on Cloudflare. Now the website feels more responsive and neater. That’s a good starting point to (re)start blogging!

Leave a Comment

Docker Postgres “PANIC: could not locate a valid checkpoint record”

Published 2017/09/14 by Daoyuan Li

It seemed that my Postgres database was not properly shut down when rebooting and when I tried to use docker-compose to start it again, the following message was shown in docker logs:

To fix this, first shut down this container (docker-compose down), then start the container in interactive mode:

After the transaction log is reset, everything should be fine. Now you can start your containers again (docker-compose up -d).

Leave a Comment

Everything is bigger here

Published 2016/07/02 by Daoyuan Li

During my first one and half days days in the US, or more specifically, Redwood City in SF bay area, it seems that I’m not having any resonance with the local people’s esthetical  tastes: I maybe wrong, but I think their surroundings basically suggest “the bigger the better”.

Before coming t0 the US, my daily supervisor said his impression of the Silicon Vally is just very big highways. Indeed as I went out the airport, it’s difficult to ignore the wide 8 or 10 lane motorway (or freeway as shown on traffic signs). But it’s not just the highway that’s bigger than that in China or Europe: cars are bigger, traffic signs are bigger, parking space is bigger, yards are bigger, hotel rooms are bigger, beds are bigger and higher, chairs are bigger, drawers are bigger, even AC and TC remotes and their buttons are bigger, wild grass are higher… Maybe I’m prejudiced, these bigger objects are not necessarily beautiful, instead they look weird to my eyes. Delicacy seems to be from another world. Wider highway does not seem to make the traffic flow better, roads are wide but not flatly paved, pedestrian ways are wide but are sometimes only available on one side…

Looks like I’m not appreciating the locals’ tastes…

Leave a Comment

Exporting and Importing Elasticsearch Indicies

Published 2016/02/17 by Daoyuan Li

In my project I need to run some local tests with data from a production elasticsearch cluster, so I exported data from the production server and imported to my local cluster. This can also be used when backing up and restoring data. Here’re the instructions.

Before you start, check out the official documentation: Snapshot and Restore.

Backing up/exporting data:

  1. Modify your eleasticsearch configuration file (normally elasticsearch.yml) and add a path.repo line, for example:
  2. Make sure this path has the correct permissions so that elasticsearch can read and write.
  3. Create snapshot:
  4. Copy the files in the configured location to your local machine.

Restoring/importing data:

  1. Modify your local elasticsearch configuration similarly like step 1 when backing up.
  2. Place the snapshot files to the repo path.
  3. Close your indices:
  4. Import data:
  5. Reopen your indices:

It is important that your the elasticsearch version on your importing party is compatible with the one exporting data, i.e., in this case your local machine has to be the same version or newer. If not, you need to upgrade elasticsearch first. The official documentation says:

The information stored in a snapshot is not tied to a particular cluster or a cluster name. Therefore it’s possible to restore a snapshot made from one cluster into another cluster. All that is required is registering the repository containing the snapshot in the new cluster and starting the restore process. The new cluster doesn’t have to have the same size or topology. However, the version of the new cluster should be the same or newer than the cluster that was used to create the snapshot.

2 Comments

Installing Theano and CUDA on Mac OS X

Published 2015/10/28 by Daoyuan Li

I started trying Theano today and wanted to use the GPU (NVIDIA GeForce GT 750M 2048 MB) on my Mac. Here’s a brief instruction on how to use the GPU on Mac, largely following the instructions from http://deeplearning.net/software/theano/install.html#mac-os.

Install Theano:

Download and install CUDA: https://developer.nvidia.com/cuda-downloads

Put the following lines into your ~/.bash_profile:

Note that the PATH line is necessary. Otherwise you may see the following message:

ERROR (theano.sandbox.cuda): nvcc compiler not found on $PATH. Check your nvcc installation and try again.

Configure Theano:

Test if GPU is used:

A more realistic example:

So it seems this GPU does not outperform the CPU. Well,GT 750M may not be the best GPU you can get… Someone else here has a similar experience.

 

5 Comments

Another Mid-Autumn Festival

Published 2015/09/27 by Daoyuan Li

Five years ago I wrote “Lonely as the moon” in my blog during my last year of master’s study, feeling lonely, confused and frustrated due to the study, insufficient hope of finding a job, and most importantly, not having someone by my side whom I can talk with and listen to. I prayed to the moon in hope of finding a girlfriend, and magically it worked within a few months. Now that this girlfriend has become my wife, the emptiness still rises from deep inside.

I’ve never foreseen this coming. I thought distance wouldn’t matter to loving hearts. Maybe not much, but little by little things can go from bad to worse. Today I really felt threatened. It made me horrified to just think about the one I love could leave me, and that not living together opens harmful uncertainties. I wish I had communicated more and made her happier. I wish I had continued my job in Finland so that she didn’t have to go back to China. I wish I had appreciated her love earlier so that she doesn’t think I’m still as stubborn as I used to be. I wish I had given her everything. I wish I had focused on solving current problems and hadn’t thought about our future plans too pessimistically.

Now I have a more clear goal: just to keep her happy and feel being loved by me. I’m going to fixed what I’ve screwed up. I’ll talk equally without judgements and listen with appreciation. Offer to help but not mandate. Become more attractive, supportive, romantic, confident and cheerful. Make more money or at least explain the plans and follow them up. Care about her and let her know to make her secure. Love her. Create a family atmosphere for us.

She is my wife and I’m her man.

Today the moon is as bright as it was five years ago. I’m sure it has heard me well.

Leave a Comment

The plethora of opportunities

Published 2015/06/02 by Daoyuan Li

For most of us, we live in a society with abundant resources: delicious food to taste, delicate clothes to put on, various activities to attend to, wonderful places to visit and numerous things to do. Yet, all these abundance are not necessarily good for us: we’re getting distracted to the extend that we don’t know what exactly we’re chasing for.

Some people say we are getting superficial — that we are trying endlessly from one thing to another and shortly getting tired of each of them. But this superficialness doesn’t come from nothing. We’re superficial not because there exist many temptations, instead, we’re not able to resist these temptations that trick us into thinking about illusions and trajectories of ourselves. We try to find life meanings by doing things but often end up in vain, maybe the right way to go is not finding meanings: maybe it’s the other way round. How we do things and live our lives is simply the meaning.

The meaning of a nomad’s life is traveling and not settling anywhere. The life meaning of a adventurer is adventure. A person focusing constantly on something becomes an expert.

Opportunities are always there, but we are not. It’s not an opportunity without the capability of taking advantage of it, be it the accumulation of prior experiences or fast learning and adapting abilities.

Indeed there are too many opportunities, however, one does not need thousands of opportunities to succeed. One is enough. Prepare for it, focus on it, reach for it before it comes to you and let it work for you.

Leave a Comment

Migrating/Installing Prestashop to Amazon AWS EC2

Published 2015/02/25 by Daoyuan Li

Recently I migrated a Prestashop website to AWS/EC2 and the process is quite smooth. Here’s how to do it.

1. Get your Website data: including Prestashop website files and a latest database dump. Remove all the files except index.php in /cache/smarty/compile and /cache/smarty/cache. (See here.)

2. Launch your EC2 instance. But before doing that, check first if a pre-configured or managed Prestashop is available in your preferred region (more info here). If not, choose the appropriate instance type and get it running. In my case I chose Ubuntu Linux.

3. Install the AMP (Apache, MySQL and PHP). On Ubuntu you can simply do it with a few simple commands:

Make sure that you remember your password for MySQL root user and phpmyadmin user.

4. Copy your database dump and website files to your EC2 instance using scp, for instance:

5. Create a MySQL user for Prestashop, you don’t want to use the root user for this purpose. You can do this using phpmyadmin or a command line. Also, create a database for Prestashop and grant the user you’ve created just now all privileges on this database.

6. Import your database from the dump:

7. Extract your website files on your EC2 instance. In my case, inside /var/www/html/. If there is an index.html file inside this directory, remove it first. Make sure you set the correct user and group to all the website files:

Note that if don’t set the correct permissions, you may get complaints from Apache, like the following:

PHP Fatal error:  Uncaught exception ‘SmartyException’ with message ‘unable to write file /var/www/html/cache/smarty/compile/46/d9/69/wrt54ecd6c1856a72.61943173’ in /var/www/html/tools/smarty/sysplugins/smarty_internal_write_file.php:44

Stack trace:

#0 /var/www/html/tools/smarty/sysplugins/smarty_internal_template.php(201): Smarty_Internal_Write_File::writeFile(‘/var/www/html/c…’, ‘<?php /* Smarty…’, Object(Smarty))

#1 /var/www/html/tools/smarty/sysplugins/smarty_internal_templatebase.php(155): Smarty_Internal_Template->compileTemplateSource()

#2 /var/www/html/classes/controller/AdminController.php(1936): Smarty_Internal_TemplateBase->fetch(‘controllers/mod…’)

#3 /var/www/html/classes/controller/AdminController.php(2337): AdminControllerCore->initModal()

#4 /var/www/html/classes/controller/Controller.php(163): AdminControllerCore->init()

#5 /var/www/html/classes/Dispatcher.php(373): ControllerCore->run()

#6 /var/www/html/admin123/index.php(54): DispatcherCore->dispatch()

#7 {main}\n  thrown in /var/www/html/tools/smarty/sysplugins/smarty_internal_write_file.php on line 44

8. Enable mcrypt for PHP:

9. Update your Prestashop settings in the config/settings.inc.php file. Set the correct database information.

10. Enable mod_rewrite:

Also, make sure to add something like the following in your apache site configuration file:

And then remember to restart your apache server.

11. Install sendmail:

12. Go to your Prestashop back office and reconfigure your domain. Go to Preferences -> SEO & URLs and scroll down to Set shop URL section.

Now everything should be OK, your new Prestashop website is available on your new domain!

If you have any issues, just drop a line here or email me.

4 Comments

Luxembourg at first glance

Published 2015/01/21 by Daoyuan Li

Now it’s been two weeks since I arrived at Luxembourg. In general the experience has been positive, except that it’s been raining for a consecutive of ten days and snowing for the rest days. It’s a bit humid,but not to the extend that I should complain. At least the air is free of smoke and particles.

It’s quite where I live, although it’s just fifteen minutes walk to the old town as well as the railway station. Almost everything is in walking distance, supermarkets, banks, hospitals, schools, government offices,Asian markets. There’s also a police school just 200 meters away. I suppose it’s a safe neighborhood.

Work is also nice, though one has to be selfmotivated. Coworkers are extremely nice, but many of them prefer speaking French, which I believe is good for me since I’d like to pick up the language. I already applied for a university language course. There’s also opportunities to meet great minds. This morning there was a distinguished lecture, where one professor talked about privacy in internet using big data.

The startup business is going slowly but according to the plan. Office isn’t too far away and fits our purpose. Orders for office computers are placed this evening and we can start the real work shortly. It’s good to have someone supporting the business, with both funding and experience.

One thing brothers me is that the washing machine is broken and it’s talking forever to fix it. The agent promises anything but I’ve now learned to be wiser and not to believe her words.

Life is much easier when you have someone helping you out. For that I’m extremely grateful for the help I received from my friend.

And I miss my girlfriend. It’s a journey we’ve chosen together, and there has to be temporary sacrifices for a better tomorrow. So for now just follow what we’ve planned. I believe we’re heading for the better.

Leave a Comment

OSX Yosemite cannot boot when TRIM is enabled on third-party SSD

Published 2014/10/18 by Daoyuan Li

After upgrading my MBP to OSX Yosemite and enabling TRIM, the computer cannot boot into the system after restart. It stuck with a no-entry sign.

You can solve this issue by following instructions from http://www.cindori.org/trim-in-os-x-yosemite/

Boot your Mac in Recovery Mode by holding Cmd-R during boot

Open the Terminal from the menu bar

Run these commands, replacing YourDisk with the name of your Mac disk

rm -rf /Volumes/YourDisk/System/Library/Extensions/IOAHCIFamily.kext
cp -r /System/Library/Extensions/IOAHCIFamily.kext /Volumes/YourDisk/System/Library/Extensions/IOAHCIFamily.kext
touch /Volumes/YourDisk/System/Library/Extensions
kextcache -u /Volumes/YourDisk

For me, I have to first use the disk utility to unlock my SSD first since I have Filevault enabled, otherwise the first command above will complain about “readonly” files.

Since TRIM is very helpful for SSDs, it’s encouraged to have it enabled for better performance. You can use Trim Enabler v3.2.5 and above without issues. But note that it disables kext signing on your mac, “it still leaves you with the same amount of security as in OS X Mavericks, where the kext signing requirement didn’t exist”. (http://www.cindori.org/update-on-trim-in-yosemite/)

4 Comments