Tag: Mac OS X

TeXLive install all packages using command line tlmgr

Published 2017/09/17 by Daoyuan Li

sudo tlmgr install scheme-full

See: https://www.tug.org/texlive/pkginstall.html

Installing Theano and CUDA on Mac OS X

Published 2015/10/28 by Daoyuan Li

I started trying Theano today and wanted to use the GPU (NVIDIA GeForce GT 750M 2048 MB) on my Mac. Here’s a brief instruction on how to use the GPU on Mac, largely following the instructions from http://deeplearning.net/software/theano/install.html#mac-os.

Install Theano:

$ pip install Theano

Download and install CUDA: https://developer.nvidia.com/cuda-downloads

Put the following lines into your ~/.bash_profile:

# Theano and CUDA
PATH="/Developer/NVIDIA/CUDA-7.5/bin/:$PATH"
export LD_LIBRARY_PATH=/Developer/NVIDIA/CUDA-7.5/lib/
export CUDA_ROOT=/Developer/NVIDIA/CUDA-7.5/
export THEANO_FLAGS='mode=FAST_RUN,device=gpu,floatX=float32'

Note that the PATH line is necessary. Otherwise you may see the following message:

ERROR (theano.sandbox.cuda): nvcc compiler not found on $PATH. Check your nvcc installation and try again.

Configure Theano:

$ cat .theanorc 
[gcc]
cxxflags = -L/usr/local/lib -L/Developer/NVIDIA/CUDA-7.5/lib/

Test if GPU is used:

$ cat check.py 
from theano import function, config, shared, sandbox
import theano.tensor as T
import numpy
import time

vlen = 10 * 30 * 768  # 10 x #cores x # threads per core
iters = 1000

rng = numpy.random.RandomState(22)
x = shared(numpy.asarray(rng.rand(vlen), config.floatX))
f = function([], T.exp(x))
print(f.maker.fgraph.toposort())
t0 = time.time()
for i in xrange(iters):
    r = f()
t1 = time.time()
print("Looping %d times took %f seconds" % (iters, t1 - t0))
print("Result is %s" % (r,))
if numpy.any([isinstance(x.op, T.Elemwise) for x in f.maker.fgraph.toposort()]):
    print('Used the cpu')
else:
    print('Used the gpu')

$ THEANO_FLAGS=mode=FAST_RUN,device=cpu,floatX=float32 time python check.py 
[Elemwise{exp,no_inplace}(<TensorType(float32, vector)>)]
Looping 1000 times took 1.743682 seconds
Result is [ 1.23178029  1.61879337  1.52278066 ...,  2.20771813  2.29967761
  1.62323284]
Used the cpu
        2.47 real         2.19 user         0.27 sys
$ THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 time python check.py 
Using gpu device 0: GeForce GT 750M
[GpuElemwise{exp,no_inplace}(<CudaNdarrayType(float32, vector)>), HostFromGpu(GpuElemwise{exp,no_inplace}.0)]
Looping 1000 times took 1.186971 seconds
Result is [ 1.23178029  1.61879349  1.52278066 ...,  2.20771813  2.29967761
  1.62323296]
Used the gpu
        2.09 real         1.59 user         0.41 sys

A more realistic example:

$ cat lr.py 
import numpy
import theano
import theano.tensor as T
rng = numpy.random

N = 400
feats = 784
D = (rng.randn(N, feats).astype(theano.config.floatX),
rng.randint(size=N,low=0, high=2).astype(theano.config.floatX))
training_steps = 10000

# Declare Theano symbolic variables
x = T.matrix("x")
y = T.vector("y")
w = theano.shared(rng.randn(feats).astype(theano.config.floatX), name="w")
b = theano.shared(numpy.asarray(0., dtype=theano.config.floatX), name="b")
x.tag.test_value = D[0]
y.tag.test_value = D[1]

# Construct Theano expression graph
p_1 = 1 / (1 + T.exp(-T.dot(x, w)-b)) # Probability of having a one
prediction = p_1 > 0.5 # The prediction that is done: 0 or 1
xent = -y*T.log(p_1) - (1-y)*T.log(1-p_1) # Cross-entropy
cost = xent.mean() + 0.01*(w**2).sum() # The cost to optimize
gw,gb = T.grad(cost, [w,b])

# Compile expressions to functions
train = theano.function(
            inputs=[x,y],
            outputs=[prediction, xent],
            updates=[(w, w-0.01*gw), (b, b-0.01*gb)],
            name = "train")
predict = theano.function(inputs=[x], outputs=prediction,
            name = "predict")

if any([x.op.__class__.__name__ in ['Gemv', 'CGemv', 'Gemm', 'CGemm'] for x in
        train.maker.fgraph.toposort()]):
    print('Used the cpu')
elif any([x.op.__class__.__name__ in ['GpuGemm', 'GpuGemv'] for x in
          train.maker.fgraph.toposort()]):
    print('Used the gpu')
else:
    print('ERROR, not able to tell if theano used the cpu or the gpu')
    print(train.maker.fgraph.toposort())

for i in range(training_steps):
    pred, err = train(D[0], D[1])

print("target values for D")
print(D[1])

print("prediction on D")
print(predict(D[0]))
$ THEANO_FLAGS=mode=FAST_RUN,device=cpu,floatX=float32 time python lr.py 
Used the cpu
target values for D
[ 1.  1.  0.  1.  0.  0.  0.  0.  0.  1.  1.  0.  0.  0.  0.  0.  0.  1.
  1.  0.  0.  1.  0.  0.  1.  1.  0.  1.  1.  1.  1.  0.  1.  1.  0.  1.
  0.  0.  0.  0.  0.  1.  0.  0.  0.  1.  1.  0.  1.  1.  1.  0.  1.  0.
  0.  0.  0.  0.  0.  1.  0.  1.  0.  0.  0.  1.  1.  1.  0.  0.  1.  1.
  1.  1.  0.  0.  0.  1.  0.  0.  1.  1.  0.  0.  1.  1.  1.  1.  0.  1.
  0.  0.  0.  0.  1.  0.  0.  1.  1.  1.  0.  0.  1.  1.  1.  1.  1.  1.
  1.  1.  1.  1.  0.  1.  1.  0.  0.  1.  0.  0.  0.  1.  0.  1.  1.  1.
  1.  0.  0.  1.  0.  1.  1.  1.  1.  1.  1.  1.  1.  1.  0.  1.  1.  0.
  1.  0.  1.  1.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  1.  0.  0.
  1.  0.  1.  0.  0.  1.  0.  0.  1.  1.  1.  1.  0.  1.  0.  0.  1.  0.
  0.  0.  1.  1.  1.  1.  1.  1.  1.  0.  1.  1.  1.  0.  1.  0.  1.  0.
  0.  1.  1.  0.  0.  1.  0.  0.  0.  0.  0.  0.  0.  1.  0.  1.  0.  1.
  1.  0.  1.  1.  1.  0.  0.  1.  1.  1.  1.  0.  0.  0.  1.  1.  0.  0.
  1.  0.  0.  0.  0.  1.  1.  1.  0.  1.  1.  1.  0.  1.  0.  0.  0.  0.
  0.  1.  1.  1.  1.  1.  1.  0.  0.  1.  1.  1.  0.  1.  0.  1.  0.  1.
  1.  0.  0.  0.  1.  1.  0.  0.  1.  0.  0.  0.  0.  1.  0.  0.  0.  1.
  0.  1.  0.  1.  1.  0.  1.  1.  0.  0.  0.  0.  1.  0.  0.  0.  0.  1.
  0.  1.  0.  0.  1.  1.  0.  0.  1.  1.  0.  1.  0.  1.  0.  0.  1.  1.
  0.  1.  1.  0.  0.  1.  1.  0.  0.  1.  0.  1.  1.  0.  0.  0.  1.  0.
  0.  0.  1.  0.  0.  0.  0.  1.  1.  0.  1.  1.  1.  0.  1.  1.  1.  1.
  1.  0.  0.  1.  0.  0.  0.  0.  1.  1.  0.  0.  0.  0.  0.  1.  1.  1.
  0.  1.  1.  1.  0.  0.  0.  0.  1.  1.  1.  0.  0.  0.  0.  1.  0.  0.
  1.  1.  0.  1.]
prediction on D
[1 1 0 1 0 0 0 0 0 1 1 0 0 0 0 0 0 1 1 0 0 1 0 0 1 1 0 1 1 1 1 0 1 1 0 1 0
 0 0 0 0 1 0 0 0 1 1 0 1 1 1 0 1 0 0 0 0 0 0 1 0 1 0 0 0 1 1 1 0 0 1 1 1 1
 0 0 0 1 0 0 1 1 0 0 1 1 1 1 0 1 0 0 0 0 1 0 0 1 1 1 0 0 1 1 1 1 1 1 1 1 1
 1 0 1 1 0 0 1 0 0 0 1 0 1 1 1 1 0 0 1 0 1 1 1 1 1 1 1 1 1 0 1 1 0 1 0 1 1
 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 1 0 0 1 0 0 1 1 1 1 0 1 0 0 1 0 0 0 1 1 1
 1 1 1 1 0 1 1 1 0 1 0 1 0 0 1 1 0 0 1 0 0 0 0 0 0 0 1 0 1 0 1 1 0 1 1 1 0
 0 1 1 1 1 0 0 0 1 1 0 0 1 0 0 0 0 1 1 1 0 1 1 1 0 1 0 0 0 0 0 1 1 1 1 1 1
 0 0 1 1 1 0 1 0 1 0 1 1 0 0 0 1 1 0 0 1 0 0 0 0 1 0 0 0 1 0 1 0 1 1 0 1 1
 0 0 0 0 1 0 0 0 0 1 0 1 0 0 1 1 0 0 1 1 0 1 0 1 0 0 1 1 0 1 1 0 0 1 1 0 0
 1 0 1 1 0 0 0 1 0 0 0 1 0 0 0 0 1 1 0 1 1 1 0 1 1 1 1 1 0 0 1 0 0 0 0 1 1
 0 0 0 0 0 1 1 1 0 1 1 1 0 0 0 0 1 1 1 0 0 0 0 1 0 0 1 1 0 1]
        8.92 real         8.24 user         1.14 sys
$ THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 time python lr.py 
Using gpu device 0: GeForce GT 750M
Used the gpu
target values for D
[ 1.  0.  0.  0.  0.  1.  0.  0.  1.  1.  0.  0.  1.  1.  0.  0.  1.  1.
  0.  0.  0.  1.  1.  0.  1.  1.  1.  0.  0.  1.  1.  1.  1.  1.  1.  0.
  0.  1.  0.  0.  1.  1.  0.  0.  1.  1.  0.  1.  0.  1.  1.  0.  1.  1.
  1.  0.  1.  1.  0.  0.  0.  1.  1.  1.  1.  1.  0.  0.  1.  1.  0.  1.
  1.  1.  1.  0.  1.  1.  0.  1.  1.  1.  0.  0.  0.  1.  1.  0.  0.  0.
  1.  0.  1.  0.  0.  0.  0.  1.  1.  1.  1.  0.  0.  1.  0.  1.  0.  1.
  1.  0.  1.  1.  0.  0.  0.  0.  1.  0.  0.  1.  0.  0.  0.  1.  0.  1.
  1.  1.  0.  0.  0.  1.  0.  1.  0.  1.  0.  1.  1.  1.  1.  1.  0.  1.
  1.  0.  1.  1.  0.  0.  1.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  0.
  1.  0.  0.  1.  1.  1.  1.  0.  0.  0.  1.  1.  1.  0.  1.  0.  0.  1.
  1.  1.  1.  1.  1.  1.  1.  1.  0.  0.  0.  0.  0.  1.  1.  1.  0.  1.
  0.  1.  0.  1.  1.  1.  1.  0.  0.  0.  1.  1.  1.  1.  0.  0.  0.  1.
  0.  1.  1.  1.  0.  1.  1.  1.  0.  0.  0.  0.  1.  0.  1.  0.  0.  1.
  0.  0.  1.  1.  0.  1.  0.  1.  1.  1.  0.  0.  1.  1.  0.  0.  0.  0.
  1.  0.  0.  1.  0.  0.  0.  0.  1.  0.  0.  1.  1.  1.  1.  1.  1.  1.
  0.  1.  1.  0.  0.  0.  1.  0.  1.  1.  0.  0.  0.  0.  0.  0.  1.  0.
  1.  1.  1.  0.  0.  1.  0.  1.  0.  0.  1.  0.  1.  0.  0.  1.  0.  0.
  1.  1.  0.  1.  1.  1.  0.  0.  0.  0.  0.  1.  0.  1.  0.  0.  0.  1.
  0.  0.  1.  1.  0.  1.  1.  0.  1.  1.  1.  0.  1.  1.  0.  0.  0.  0.
  0.  0.  1.  1.  1.  1.  1.  1.  1.  1.  0.  1.  1.  1.  0.  1.  0.  1.
  1.  1.  0.  1.  1.  0.  1.  1.  1.  0.  0.  1.  1.  0.  0.  0.  0.  0.
  1.  0.  0.  1.  1.  1.  0.  1.  0.  0.  1.  1.  0.  1.  1.  0.  1.  1.
  0.  0.  1.  0.]
prediction on D
[1 0 0 0 0 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 0 1 1 0 1 1 1 0 0 1 1 1 1 1 1 0 0
 1 0 0 1 1 0 0 1 1 0 1 0 1 1 0 1 1 1 0 1 1 0 0 0 1 1 1 1 1 0 0 1 1 0 1 1 1
 1 0 1 1 0 1 1 1 0 0 0 1 1 0 0 0 1 0 1 0 0 0 0 1 1 1 1 0 0 1 0 1 0 1 1 0 1
 1 0 0 0 0 1 0 0 1 0 0 0 1 0 1 1 1 0 0 0 1 0 1 0 1 0 1 1 1 1 1 0 1 1 0 1 1
 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 0 1 1 1 1 0 0 0 1 1 1 0 1 0 0 1 1 1 1 1 1
 1 1 1 0 0 0 0 0 1 1 1 0 1 0 1 0 1 1 1 1 0 0 0 1 1 1 1 0 0 0 1 0 1 1 1 0 1
 1 1 0 0 0 0 1 0 1 0 0 1 0 0 1 1 0 1 0 1 1 1 0 0 1 1 0 0 0 0 1 0 0 1 0 0 0
 0 1 0 0 1 1 1 1 1 1 1 0 1 1 0 0 0 1 0 1 1 0 0 0 0 0 0 1 0 1 1 1 0 0 1 0 1
 0 0 1 0 1 0 0 1 0 0 1 1 0 1 1 1 0 0 0 0 0 1 0 1 0 0 0 1 0 0 1 1 0 1 1 0 1
 1 1 0 1 1 0 0 0 0 0 0 1 1 1 1 1 1 1 1 0 1 1 1 0 1 0 1 1 1 0 1 1 0 1 1 1 0
 0 1 1 0 0 0 0 0 1 0 0 1 1 1 0 1 0 0 1 1 0 1 1 0 1 1 0 0 1 0]
       19.78 real        17.61 user         1.24 sys

So it seems this GPU does not outperform the CPU. Well,GT 750M may not be the best GPU you can get… Someone else here has a similar experience.

5 Comments

OSX Yosemite cannot boot when TRIM is enabled on third-party SSD

Published 2014/10/18 by Daoyuan Li

After upgrading my MBP to OSX Yosemite and enabling TRIM, the computer cannot boot into the system after restart. It stuck with a no-entry sign.

You can solve this issue by following instructions from http://www.cindori.org/trim-in-os-x-yosemite/

Boot your Mac in Recovery Mode by holding Cmd-R during boot
Open the Terminal from the menu bar
Run these commands, replacing YourDisk with the name of your Mac disk
rm -rf /Volumes/YourDisk/System/Library/Extensions/IOAHCIFamily.kext
cp -r /System/Library/Extensions/IOAHCIFamily.kext /Volumes/YourDisk/System/Library/Extensions/IOAHCIFamily.kext
touch /Volumes/YourDisk/System/Library/Extensions
kextcache -u /Volumes/YourDisk

For me, I have to first use the disk utility to unlock my SSD first since I have Filevault enabled, otherwise the first command above will complain about “readonly” files.

Since TRIM is very helpful for SSDs, it’s encouraged to have it enabled for better performance. You can use Trim Enabler v3.2.5 and above without issues. But note that it disables kext signing on your mac, “it still leaves you with the same amount of security as in OS X Mavericks, where the kext signing requirement didn’t exist”. (http://www.cindori.org/update-on-trim-in-yosemite/)

4 Comments

Debugging Apache configuration files on Mac OS X

Published 2013/10/31 by Daoyuan Li

Sometimes I make some modification to my site’s apache config and apache stops functioning, but I cannot get enough information from /var/log/apache2/error.log. Here’s a command that will tell you where the problem is:

sudo /usr/sbin/httpd -k start -e Debug -E /dev/stdout

It starts apache for debugging and prints out the messages on screen, so that hopefully you can find something useful.

Leave a Comment

mod_wsgi and mod_xsendfile on OS X 10.9 Mavericks

Published 2013/10/27 by Daoyuan Li

Updated on Nov 4, 2013: The following tricks may still work, however I have found a much easier solution. Simply install Xcode command line developer tools and you should be able to compile source code without issues:

$ xcode-select --install

After upgrading my Mac from 10.8 to 10.9 Mavericks my apache stopped working, so I have to reinstall mod_wsgi and mod_xsendfile. However, tricks are needed to compile and install these mods successfully.

Upgrade Xcode

Upgrade Xcode in App Store. As mentioned by Valerie:

I had to manually upgrade Xcode (after Mavericks upgrade) from the App Store & agree to its license because ./configure hung forever until I did that.

mod_wsgi

For mod_wsgi installation, create a soft link to OSX10.9.xctoolchain:

cd /Applications/Xcode.app/Contents/Developer/Toolchains/
sudo ln -s XcodeDefault.xctoolchain OSX10.9.xctoolchain

Then run configure under mod_wsgi source code directory:

mod_wsgi-3.4$ ./configure

It will generate a Makefile similar as follows:

#  Copyright 2007 GRAHAM DUMPLETON                                               
#                                                                                
#  Licensed under the Apache License, Version 2.0 (the "License");               
#  you may not use this file except in compliance with the License.              
#  You may obtain a copy of the License at                                       
#                                                                                
#      http://www.apache.org/licenses/LICENSE-2.0                                
#                                                                                
#  Unless required by applicable law or agreed to in writing, software           
#  distributed under the License is distributed on an "AS IS" BASIS,             
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.      
#  See the License for the specific language governing permissions and           
#  limitations under the License.                                                

APXS = /usr/sbin/apxs                                                            
PYTHON = /usr/local/bin/python                                                   

DESTDIR =                                                                        
LIBEXECDIR = /usr/libexec/apache2                                                

CPPFLAGS =  -I/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/include/python2.7 -DNDEBUG 
CFLAGS =   -Wc,'-arch x86_64'                                                    
LDFLAGS =  -Wl,-F/usr/local/Cellar/python/2.7.5/Frameworks -framework Python -u _PyMac_Error   -arch x86_64
LDLIBS =  -ldl  -framework CoreFoundation                                        

all : mod_wsgi.la                                                                

mod_wsgi.la : mod_wsgi.c                                                         
    $(APXS) -c $(CPPFLAGS) $(CFLAGS) mod_wsgi.c $(LDFLAGS) $(LDLIBS)             

$(DESTDIR)$(LIBEXECDIR) :                                                        
    mkdir -p $@                                                                  

install : all $(DESTDIR)$(LIBEXECDIR)                                            
    $(APXS) -i -S LIBEXECDIR=$(DESTDIR)$(LIBEXECDIR) -n 'mod_wsgi' mod_wsgi.la 

clean :                                                                          
    -rm -rf .libs                                                                
    -rm -f mod_wsgi.o mod_wsgi.la mod_wsgi.lo mod_wsgi.slo mod_wsgi.loT          
    -rm -f config.log config.status                                              
    -rm -rf autom4te.cache                                                       

distclean : clean                                                                
    -rm -f Makefile Makefile.in                                                  

realclean : distclean                                                            
    -rm -f configure

However, this Makefile is not correct and running ‘make’ the compiler will complain something like:

mod_wsgi.c:34:10: fatal error: 'httpd.h' file not found

Prepend the following line to CPPFLAGS value:

-I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/apr-1 -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/apache2

Save the Makefile and it will look something like:

#  Copyright 2007 GRAHAM DUMPLETON                                               
#                                                                                
#  Licensed under the Apache License, Version 2.0 (the "License");               
#  you may not use this file except in compliance with the License.              
#  You may obtain a copy of the License at                                       
#                                                                                
#      http://www.apache.org/licenses/LICENSE-2.0                                
#                                                                                
#  Unless required by applicable law or agreed to in writing, software           
#  distributed under the License is distributed on an "AS IS" BASIS,             
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.      
#  See the License for the specific language governing permissions and           
#  limitations under the License.                                                

APXS = /usr/sbin/apxs                                                            
PYTHON = /usr/local/bin/python                                                   

DESTDIR =                                                                        
LIBEXECDIR = /usr/libexec/apache2                                                

CPPFLAGS =  -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/apr-1 -I/Applications/Xcode. app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/apache2 -I/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/include/python2.7 -DNDEBUG
CFLAGS =   -Wc,'-arch x86_64'                                                             
LDFLAGS =  -Wl,-F/usr/local/Cellar/python/2.7.5/Frameworks -framework Python -u _PyMac_Error   -arch x86_64
LDLIBS =  -ldl  -framework CoreFoundation                                        

all : mod_wsgi.la                                                                

mod_wsgi.la : mod_wsgi.c                                                         
    $(APXS) -c $(CPPFLAGS) $(CFLAGS) mod_wsgi.c $(LDFLAGS) $(LDLIBS)             

$(DESTDIR)$(LIBEXECDIR) :                                                        
    mkdir -p $@                                                                  

install : all $(DESTDIR)$(LIBEXECDIR)                                            
    $(APXS) -i -S LIBEXECDIR=$(DESTDIR)$(LIBEXECDIR) -n 'mod_wsgi' mod_wsgi.la 

clean :                                                                          
    -rm -rf .libs                                                                
    -rm -f mod_wsgi.o mod_wsgi.la mod_wsgi.lo mod_wsgi.slo mod_wsgi.loT          
    -rm -f config.log config.status                                              
    -rm -rf autom4te.cache                                                       

distclean : clean                                                                
    -rm -f Makefile Makefile.in                                                  

realclean : distclean                                                            
    -rm -f configure

Then make && install:

mod_wsgi-3.4$ make
mod_wsgi-3.4$ sudo make install

Modify /etc/apache2/httpd.conf to enable mod_wsgi:

LoadModule wsgi_module libexec/apache2/mod_wsgi.so

mod_xsendfile

Use the following command to compile and install mod_xsendfile:

sudo apxs -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/apr-1 -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/apache2 -cia mod_xsendfile.c

Enabling HTTPS/SSL

After upgrading OS X the apache configuration was reset, but your original config is save to /etc/apache2/httpd.conf.pre-update. I need to enable SSL in httpd.conf again by uncommenting the following line:

# Include /private/etc/apache2/extra/httpd-ssl.conf

Restart Apache and everything should work fine

sudo apachectl restart

3 Comments

Solution: dd too slow on Mac OS X

Published 2013/10/09 by Daoyuan Li

When I was cloning SD cards on Mac OS X using `dd’, it takes ages to get things done. I was using the following command:

diskutil unmountDisk /dev/disk2
sudo dd bs=1m if=~/Downloads/2013-10-09.alice.img of=/dev/disk2

It takes much less time when using /dev/rdisk2 instead of /dev/disk2:

diskutil unmountDisk /dev/disk2
sudo dd bs=1m if=~/Downloads/2013-10-09.alice.img of=/dev/rdisk2

The reason is that rdisks are “raw” thus resulting in a higher R/W speed, according to `man hdiutil` [1]:

/dev/rdisk nodes are character-special devices, but are “raw” in the BSD sense and force block-aligned I/O. They are closer to the physical disk than the buffer cache. /dev/disk nodes, on the other hand, are buffered block-special devices and are used primarily by the kernel’s filesystem code.

[1] http://superuser.com/questions/631592/mac-osx-why-is-dev-rdisk-20-times-faster-than-dev-disk

58 Comments

A “normal” sed on Mac

Published 2013/07/08 by Daoyuan Li

The `sed` program on Mac is not a standard (GNU) one. To get the normal one, use brew:

brew install gnu-sed

After this, alter PATH. For example, add the following line to your `~/.bash_profile`:

PATH="/usr/local/opt/gnu-sed/libexec/gnubin:$PATH"

And now you have a normal sed!

2 Comments

A faster key repeat rate for Mac OS X

Published 2013/07/04 by Daoyuan Li

defaults write NSGlobalDomain KeyRepeat -int 1

The fastest one can get via System Preferences is 2. You can set it to either 0 or 1, depending on your preferences.

To check your current repeat rate, issue the following command in Terminal:

defaults read NSGlobalDomain KeyRepeat

See: http://hints.macworld.com/article.php?story=20090823193018149

Leave a Comment

Installing mod_xsendfile on OS X Lion

Published 2012/11/14 by Daoyuan Li

First download mod_xsendfile.c from https://tn123.org/mod_xsendfile/

Then compile and install it:

sudo apxs -cia mod_xsendfile.c

Add the following line to /etc/apache2/httpd.conf at the end of the first block of “Load” statements:

LoadModule xsendfile_module libexec/apache2/mod_xsendfile.so

At last, restart the apache server

sudo apachectl restart

Hi, the company I’m working for (yabroad.com) is hiring Website Backend and Frontend Developers to our platform team. We are building an open platform for youngsters to travel beyond boarders and we offer youngsters internship, language study, travel and volunteer opportunities. Please contact me if you are interested.

Leave a Comment

XBMC and OS X firewall permission popups

Published 2012/07/06 by Daoyuan Li

I installed XBMC on my Mac OS X Lion and every time I open up the program, a firewall permission dialog pops up. It didn’t help even if I added the XBMC in my firewall configurations. Popups still show up. This link solves the problem neatly — You can generate a certificate and sign the program by yourself. After that the frustrating dialog should appear only once (at most). In case the link becomes invalid, here’s a screenshot of the instructions. Click to see the large picture.

Leave a Comment

Block unauthorized ssh login attempts (attacks)

Published 2012/06/30 by Daoyuan Li

My friends and I am working on a hobby project and we need a Git server, so I set up one on my MacBook Pro. We access the repository via SSH. However when I checked the ssh log I found someone trying to get ssh access to my machine by guessing usernames. The log looks as follows:

Jun 29 21:06:52 doh1 sshd[19400]: Invalid user postgres from 190.181.132.70
Jun 29 21:06:52 doh1 sshd[19401]: input_userauth_request: invalid user postgres
Jun 29 21:06:52 doh1 sshd[19401]: Received disconnect from 190.181.132.70: 11: Bye Bye
Jun 29 21:06:54 doh1 sshd[19402]: reverse mapping checking getaddrinfo for wimax132-70.yota.com.ni [190.181.132.70] failed - POSSIBLE BREAK-IN ATTEMPT!
Jun 29 21:06:54 doh1 sshd[19403]: Received disconnect from 190.181.132.70: 11: Bye Bye
Jun 29 21:06:55 doh1 sshd[19405]: reverse mapping checking getaddrinfo for wimax132-70.yota.com.ni [190.181.132.70] failed - POSSIBLE BREAK-IN ATTEMPT!
Jun 29 21:06:55 doh1 sshd[19405]: Invalid user backup from 190.181.132.70
Jun 29 21:06:55 doh1 sshd[19406]: input_userauth_request: invalid user backup
Jun 29 21:06:56 doh1 sshd[19406]: Received disconnect from 190.181.132.70: 11: Bye Bye
Jun 29 21:06:57 doh1 sshd[19407]: reverse mapping checking getaddrinfo for wimax132-70.yota.com.ni [190.181.132.70] failed - POSSIBLE BREAK-IN ATTEMPT!

I first tried to use DenyHosts, however, there are still attempts from other IP addresses. Since there are three of us accessing the repository, I configured the hosts.allow and hosts.deny manually: deny all hosts other than the IP addresses I trust.

hosts.deny:

~$ cat /etc/hosts.deny 
sshd: ALL

hosts.allow:

~$ cat /etc/hosts.allow
sshd: [The IP addresses you allow to connect via SSH]
ALL: localhost

Now the log file should be quite…

Leave a Comment

GitHub couples

Published 2012/05/21 by Daoyuan Li

I’m feeling good today because of theses things:

My mobile phone ran out of battery and the alarm clock didn’t ring this morning, but I still managed to get up just in time and caught the bus at the last minute — and arrived at the company at my usual time.
My manager told me it looks positive to renew my contract and hopefully it will be one and half years. He also says he tries to get it done before the summer vacation, which makes my life a lot easier. Also he says it’s possible to save my holidays till winter. So I’ll be back in China for some time in winter this year.
A very old lady managed to stop and got on the bus even though she waved her hand a bit late to the bus driver. The bus driver was polite and that what I like about Finland: people generally don’t get angry.
Here’s one very funny and geeky picture I saw from xda-developers. In case the link gets invalidated later, the picture reads: “So, where did you two meet?” “Windows users: at the office” “Mac users: at Starbucks” “Linux users: GitHub”.

Leave a Comment

Downgrade Preview in OS X Lion

Published 2012/04/23 by Daoyuan Li

I’ve had enough with the slow Preview application in Lion. After some digging I found this tutorial to downgrade the Preview in Lion to the Snow Leopard version. Basically you need to have a copy of both the Preview.app and /System/Library/PrivateFrameworks/MeshKit.framework from Snow Leopard. If you need to associate the default app to open pdf/png with the downgraded Preview, you have to delete the original Preview.app.

tar -pczf Preview.app.tar.gz /Applications/Preview.app
sudo rm -rf /Applications/Preview.app

Update: after installing OS X updates you need to remove the Preview.app again and put the SL Preview.app to /Applications/ directory.

Leave a Comment

i386-jos-elf toolchain on OS X Lion

Published 2012/03/18 by Daoyuan Li

Yesterday friend and I decided to follow the MIT Operating System Engineering course together in order to get a deep understanding of OS’s. And today I started setting up the cross compling environment for the labs. At first I wanted to get the toolchain from macports, but unluckily it didn’t successfully build binutils on my Mac. As a result, I started building the toolchain from the source code, following the instructions at http://pdos.csail.mit.edu/6.828/2011/tools.html.

The programs you need for the toolchain include binutils, gcc, and gdb. For compiling gcc you also need GMP, MPFR, and MPC. The source codes are available at:

wget http://ftpmirror.gnu.org/binutils/binutils-2.21.1.tar.bz2
wget ftp://ftp.gmplib.org/pub/gmp-5.0.4/gmp-5.0.4.tar.bz2
wget http://www.mpfr.org/mpfr-current/mpfr-3.1.0.tar.bz2
wget http://www.multiprecision.org/mpc/download/mpc-0.8.2.tar.gz
wget http://ftpmirror.gnu.org/gcc/gcc-4.5.1/gcc-core-4.5.1.tar.bz2
wget http://ftpmirror.gnu.org/gdb/gdb-6.8a.tar.gz

Unzip them in a directory and build binutils, gcc, and gdb one by one.

$ tar jxvf binutils-2.21.1.tar.bz2
$ cd binutils-2.21.1
$ ./configure --target=i386-jos-elf --disable-nls && make && sudo make install

In order to build gcc, GMP, MPFR and MPC needs to be built first.

$ mkdir -p libs/install
$ mv gmp-5.0.4.tar.bz2 mpfr-3.1.0.tar.bz2 mpc-0.8.2.tar.gz libs/
$ tar jxvf gmp-5.0.4.tar.bz2
$ mkdir gmp-build; cd gmp-build
$ ../gmp-5.0.4/configure --prefix=$(cd ../install && pwd)
$ make && sudo make install
$ cd ..
$ tar jxvf mpfr-3.1.0.tar.bz2
$ mkdir mpfr-build; cd mpfr-build
$ ../mpfr-3.1.0/configure --prefix=/Users/daoyuanli/Downloads/i386-elf-gcc/libs/install/ --with-gmp=/Users/daoyuanli/Downloads/i386-elf-gcc/libs/install/
$ make && sudo make install
$ cd ..
$ tar zxvf mpc-0.8.2.tar.gz
$ mkdir mpc-build; cd mpc-build
$ ./configure --prefix=/Users/daoyuanli/Downloads/i386-elf-gcc/libs/install/ --with-gmp=/Users/daoyuanli/Downloads/i386-elf-gcc/libs/install/ --with-mpfr=/Users/daoyuanli/Downloads/i386-elf-gcc/libs/install/
$ make ; sudo make install
$ cd ..
$ tar jxvf gcc-core-4.5.1.tar.bz2
$ mkdir gcc-build; cd gcc-build
$ ../gcc-4.5.1/configure --target=i386-jos-elf --disable-nls --without-headers --with-newlib --disable-threads --disable-shared --disable-libmudflap --disable-libssp --with-gmp=/Users/daoyuanli/Downloads/i386-elf-gcc/libs/install/ --with-mpfr=/Users/daoyuanli/Downloads/i386-elf-gcc/libs/install/ --with-mpc=/Users/daoyuanli/Downloads/i386-elf-gcc/libs/install/
$ make ; sudo make install

Note that it’s essential to build gcc in a directory different from the source code directory to avoid compiling errors.

Then make gdb:

$ tar zxvf gdb-6.8a.tar.gz
$ mkdir gdb-build; cd gdb-build
$ ../gdb-6.8/./configure --target=i386-jos-elf --program-prefix=i386-jos-elf- --disable-werror
$ make && sudo make install

QEMU is available in macports:

$ sudo port install qemu

15 Comments

Port already in use issue of Arduino for Eclipse plugin

Published 2012/03/14 by Daoyuan Li

When uploading with avrdude the plugin give a error prompt saying “Port already in use” (while no other application is using the port). After clicking OK it actually uploads the program to Arduino board.

port_in_use

After reading issue #6 I check the librxtxSerial.jnilib in eclipse:

.cp$ file librxtxSerial.jnilib
librxtxSerial.jnilib: Mach-O universal binary with 4 architectures
librxtxSerial.jnilib (for architecture x86_64): Mach-O 64-bit bundle x86_64
librxtxSerial.jnilib (for architecture i386): Mach-O bundle i386
librxtxSerial.jnilib (for architecture ppc7400): Mach-O bundle ppc
librxtxSerial.jnilib (for architecture ppc64): Mach-O 64-bit bundle ppc64

While in my Arduino application the librxtxSerial.jnilib output is:

llibrxtxSerial.jnilib: Mach-O universal binary with 2 architectures
librxtxSerial.jnilib (for architecture i386): Mach-O bundle i386
librxtxSerial.jnilib (for architecture ppc): Mach-O bundle ppc

So I just copied the Arduino.app//Contents/Resources/Java/librxtxSerial.jnilib file to eclipse/configuration/org.eclipse.osgi/bundles/419/1/.cp/librxtxSerial.jnilib It works after restarting eclipse. No error prompts are shown.

Leave a Comment