Showing posts with label scripting. Show all posts
Showing posts with label scripting. Show all posts

Friday, December 18, 2015

Using SSH keys to access remote servers and git repositories

An SSH key can be used to access a virtual private server or a remote git repository without the need to enter a password every time. By sharing your public key with the remote server, your compter is authenticated as a trusted access point.


Creating SSH keys 

In Debian GNU Linux, using the Gnome desktop, you can create a private and public SSH key pair with for example the seahorse key manager. Under File / New / Secure Shell Key.

Created keys will be visible under ~/.ssh/ the private key is called id_rsa and the public key id_rsa.pub. You should only share the public key.

At the command line, you can create keys with
ssh-keygen -t rsa -C "your_email@example.com"

Virtual Private Server

I bought a virtual private server with Debian pre-installed. A public key can be added in the file ~/.ssh/authorized_keys. When connected to the server, edit the file:
vim ~/.ssh/authorized_keys
You might need to change access permission to that file as explained in this gist.

Bitbucket

Your public key can be added to your bitbucket account under manage account / security / SSH keys. This page explains how to use the SSH protocol with Bitbucket in more details.

Github

Your public key can be added to your Github account under profile / settings / SSH key. More details on how to generate and use SSH keys for github.

Then at the top of your Github repository you should see the "clone URL". Copy the SSH URL, in the form: git@github.com:yourusername/yourrepository.git
Add it as a remote origin:
git remote add origin git@github.com:yourusername/yourrepository.git
If there was already a remote repository you might need to delete it first with git remote remove origin.



The push and set the remote repository as an upstream repository:
git push --set-upstream origin master
Subsequent push can be simply made with
git push

See also

See also my other blog posts on the bash shell commands and on git commands.

Wednesday, November 25, 2015

Ruby, Perl, R, Bash

A comparison of some programming languages, couldn't add python because it's not recognised as a programming language by the Google trend website.

The trend for one keyword is relative to all other searchers over the same time period. The decreasing trend of Perl in this graph does not mean that searches for Perl decreased in absolute number. It means that the proportion of these searches to the overall Google searches was decreasing. How Trends data is adjusted.

Tuesday, October 20, 2015

Data integration with Knime and the R statistical software

I am testing the Knime software to create data pipelines. I started by installing the following extensions:
  •   KNIME Connectors for Common Databases    
  •   KNIME Interactive R Statistics Integration    

Database operations


I tried chaining the node database Row filter after database selector (containing an SQL statement of the form "select * from table"). But the query was taking ages because my source table is rather large.  I replaced the SQL statement in the node database row filter by a statement of the form "select * from table where code = 999". This time the query was much shorter.
Unlike dplyr which updates the SQL query - based on the group_by(), select(), filter() verbs - before  executing a final SQL query, it seems that Knime is executing all SQL queries one after the other.

Interaction with the R statistical program


Then I pushed the data to R. input data frame is called knime.in One issue is that most character vectors are transformed into factors. This was causing various errors. max(year) returned an error, and various merge operation were failing. I had to tell R to change back all those column types to character or numeric.

I wanted to use a filter before using a plot. But I needed to filter on 2 columns. I didn't know how to implement this in Knime. A Google search returned this forum. Rule based row filter seems to work.




In the workflow above, I used R View to display  a plot generated with ggplot.

Workflow are a nice way to display data integration steps and probably easy to explain to others. Node configuration is rather straightforward, once you have found the right node in the repository. I haven't figured out yet how to use input forms and flow variables.

I don't know how easy it is to maintain functional workflows on the long term.

Tuesday, March 31, 2015

Gauss commands

Comments begin "/*" end "*/" or begin "@" end "@"

    /* Comments */
    @ Comments @


Change working directory:

    chdir
 

Load data 
The filename can be either a literal or a string. If the filename is in a string variable, then the ^ (caret) operator must precede the name of the string, as in:

    filestr = "data/filename.txt";
    loadm x = ^filestr;
 

Run a script 

    run file_name;
     

Indexing matrices
See help aptech.com.gauss.13.0/doc/LF.6-DataTypes.html
The statement

    y = x[1:3,5:8];
 

Will put the intersection of the first three rows and the 
fifth through eighth columns of x into the matrix y.

Plot

plotXY(datax[.,1], datax[.,2:cols(datax)]) 
plotXY(datay[.,1], datay[.,2:cols(datay)])

Gauss resources

Basic GAUSS workshop 2002
Aptech Tutorial, running a program file

Friday, January 16, 2015

Building an R package

A package can be seen as a coherent group of functions available for future projects. Building your own package enables you to reuse and share your statistical procedures. Function parameters and examples can be documented with Roxygen to facilitate digging back into the code later on. I created my second package  based on instructions from Hadley. My package structure is composed of the following folders:
  • R/   contains R code
  • test/  contains tests
  • inst/  contains files that will be exported with the package
  • docs/  contains .Rmd documents illustrating code development steps and data analysis.
  • data/    contains data sets exported with the package
  • data-raw/    contains raw dataset and R code to extract raw data from disk and from web resources.

Code

Create a directory containing a package skeleton
devtools::create("packagename")
RStudio has a menu build / configure build tools where devtools package functions and document generation can be linked to keyboard shortcuts:
  • document CTRL + SHIFT + D 
  • build and reload CTRL + SHIFT + B
devtools::load_all() or Cmd + Shift + L, reloads all code in the package.
Add packages to the list of required packages devtools::use_package("dplyr") devtools::use_package("ggplot2", "suggests")

Data

For data I followed his recommendations in r-pkgs/data.rmd devtools::use_data(mtcars) devtools::use_data_raw() # Creates a data-raw/ folder and add it to .Rbuildignore

Tests

Example of testing for the devtools package

Bash command to build and check a package

Bash command to build a package directory:
R CMD build packagename
Bash command to check a package tarball:

R CMD check packagename_version.tar.gz
 An error log (good luck for understanding it) is visible at:
packagename.Rcheck/00check.log
Generate the documentation and check for documentation specific errors
R CMD Rd2pdf tradeflows --no-clean
The --no-clean option keeps among other files a temporary Latex which can be inspected under:
packagename.Rcheck/packagename-manual.tex

Alternatively the build and check procedure can be run in RStudio as explained above.

Thursday, January 23, 2014

GFPM

Problem: In Windows 7, I was getting UAC message for each .exe component of the GFPM model.
Fix: stop annoying UAC prompts, recommends to use the task scheduler and set highest priviledge for the task. I used it to run a GFPM task. Then created a shortcut to:
C:\Windows\System32\schtasks.exe /RUN /TN GFPM\GFPM
I called this Shortcut "Run GFPM without UAC warning". It seems to work, there were no UAC warnings anymore!

A next step would be to run GFPM under wine in linux. That would make it easier to run more than one simulation. From wine it's not going to be possible to use Excel, so I might have to start the batchfiles after the call to world.xls.

Tuesday, January 21, 2014

R commands

A list of commonly used R commands.

Remove all objects from the workspace:
rm(list=ls())

Yihui Xie wrote that "setwd() is bad, dirty, ugly." Use relative paths instead.

 

Testthat library

Run all tests in a directory:
test_dir("tests")

Monday, January 13, 2014

Msysgit makes GNU Bash available on Windows

Msysgit makes Bash available on windows. It is related to MinGW - Minimalist GNU for Windows, itself a fork of cygwin. The version I use on my system is msysgit. With this tool, some linux like command line operations can be run on windows.

 MSYS Git FAQ:
"MSys is an environment for Windows offering a Unix-type shell and a Perl interpreter. Because many parts of Git are still not builtins programmed in C, but instead shell and Perl scripts, Git for Windows needs such an environment. Therefore we ship Git for Windows with a very minimal version of MSys."
Example of commands:
Compute md5sum
md5sum filename
See also:

Thursday, January 02, 2014

GNU-Linux bash shell commands


Linux is the kernel of the operating system on top of which other programs are built. A detailed list of GNU core utilities is available under the command :
info coreutils

 Files 

determine file type and encoding
file filename
list a directory
ls
ls -R #list subdirectories recursively
ls -lh #sizes in human readable format
Find files in subdirectories of the current directory (Quotes are requited to prevent shell command expansion).
find . -name "*.pdf"
find . -mtime 0 # modified in the last 24 hours
Find files in the whole system
locate filename

File and folder compression

Decompress a file
 gunzip file.gz
How do I compress a whole directory?
tar -zcvf archive-name.tar.gz directory-name
Where
  • -z: Compress archive using gzip program
  • -c: Create archive
  • -v: Verbose i.e display progress while creating archive
  • -f: Archive File name
To extract content from the archive in the current directory
tar -zxvf archive-name.tar.gz

Rename files

For example to rename all upper-case .JPG extension into lower-case .jpg extension.
rename 's/\.JPG$/\.jpg/' *.JPG
Change file permission:
chmod a=rwx filename
chmod 777 filename 
Change file permissions recursively:
chmod 755 directoryname
Chmod instructions can be given with characters or numbers, chmod 777 or chmod a=rwx is a question of preference.
  • Some prefer 755 over 777 because giving write access to group and other users could be a security risk. 755 leaves read and execute rights to groups and other users. 755 is visible as "rwxr-xr-x" in ls -l. 
  • The default for document files on Debian seems to be chmod 644, visible as "-rw-r--r--" in ls -l.

Text files

Count the number of lines in a file
wc -l filename.txt
Count occurrences of a word in a file
grep -roh word filename.txt  | wc -w
Remove duplicated lines from a file
awk '!a[$0]++' input.txt
Search with Grep
 grep "text" file.txt
Awk tutorial, for example  filter a large file for lines that have a third field (product code) starting with 44, keep the header line:
awk -F, '$3 ~ /^44/||NR==1' nc201501.dat|less
Regexp match begining of and end of line with ^ and $.

Follow the end of a log file as it is written 
tail -f
See tab and end of line characters in a text file
cat -te filename |less

Manipulate strings in files

Replace strings
first="I love Suzy and Mary"
second="Sara"
first=${first/Suzy/$second}
Replace strings with sed

sed -i  's/pattern/replacement/g' bli.txt
sed -i  's/^.*\://g' input_file.txt # edit file in place
grep EMAIL input_file.txt |sed  's/^.*\://g' > output_file.txt
Replace strings with perl in a git repository

git grep -lz 'readcsvfromgauss'| xargs -0 perl -i'' -pE "s/readcsvfromgauss/readcsvfromgauss0/g"

PDF files

Commands based on the poppler library for PDF manipulation.
Search a text pattern in all PDF files present in a directory:
pdfgrep pattern *.pdf
 Merge multiple PDF into one:
pdfunite in-1.pdf in-2.pdf in-n.pdf out.pdf
Alternatively, pdftk can be used to merge PDF files
pdftk input1.pdf input2.pdf cat output output.pdf

Videos and audio

Install youtube-dl using pip:
 sudo pip install --upgrade youtube_dl
Download a video from youtube :
youtube-dl video_url
Download only the audio in an .mp3 format
youtube-dl --extract-audio --audio-format mp3 video_url

Users

Check your user id
id 
What group do you belong to as a user
groups
Add a user to the super users
adduser username sudo
That user needs to re-log into the shell for the change to take effect.
Add a new user
useradd username
Set a password for the new user
passwd username
Delete a user
userdel username
Show all users
getent passwd
Show all groups
getent group

System

OS release
less /etc/os-release
Disk usage
du -h
Display available space on drives
df -h
Display available RAM memory
less /proc/meminfo

Install a program
sudo apt-get install
 System name
uname -a
file /sbin/init
hostname -f
Start and quit a super user session
su
exit
Last time the system was started
last reboot 
last
Show environment variables
printenv

Job handling

List
jobs
Bring a job to the foreground
fg job_number
Run a job in the background. A command followed by an & will run in the background.

Stop a job
CTRL ^ Z
Quit a job
 CTRL ^ C
Kill a malfunctionning program:
kill process_id
Find a program id with:
ps aux
Kill a graphical program, by clicking on it:
 xkill

Users

Create a new user
adduser user_name
Temporary log in as that user
su user_name
Delete a user
userdel user_name

Secure shell

log into a remote machine
ssh user@remote_machine
Copy a local file to a file on the remote machine
scp local_file_name user@remote_machine:path_to_file/file_name
Copy a file from the remote machine to a local file
scp user@remote_machine:path_to_file/file_name  local_file_name
Copy a full directory (dmouraty) from the remote machine:
 scp -rp user@dest:/path destdirectory

Alias

alias ll="ls -lh"

Based on how can i sort du-h output by size
alias du='du -hd1 | sort -h -r'

You can place those commands in your ~/.bashrc to create a permanent alias.
bashrc:
"You may want to put all your additions into a separate file like ~/.bash_aliases, instead of adding them here directly."

.bash_profile and .bashrc

These are places where a user can turn of the system BEEP :
setterm -blength 0
.bash_profile is executed on login shell, when you login in another tty or when you access a system through ssh. .bashrc is executed on non-login shells when you open a terminal window in Gnome.

Debian Dotfiles
"Now, since bash is being invoked as a login shell (with name "-bash", a special ancient hack), it reads /etc/profile first. Then it looks in your home directory for .bash_profile, and if it finds it, it reads that."
[...] "You may have noted that .bashrc is not being read in this situation. You should therefore always have command source ~/.bashrc at the end of your .bash_profile in order to force it to be read by a login shell.  "
In .bashrc a user can set environment variables, define alias (see above).

Keyboard

bash french blogger recommended a simple shell command to change keyboard layout :
sudo loadkeys fr
fr-keyboard on Debian wiki for a more permanent system configuration and use in GUI apps. Switching between keyboads can then be done with:
setxkbmap de
setxkbmap fr

Information about the system

  • cat /proc/meminfo
  • cat /proc/cpuinfo
  • cat /etc/debian_version
  • lsb_release -a

Shortcuts

Keyboard shortcuts for bash  for example Ctrl+A to go to the beginning of a line.

Documentation

Friday, October 25, 2013

Git commands

See also my other posts labelled git:
I've started using GIT to track file modifications and I followed this advice to set it up:  create a repository.
I use git both on windows and on Ubuntu / Debian GNU-Linux.

The commands I've used to upload content to github.com/paul4forest/forestproductsdemand are:
git remote add origin https://github.com/paul4forest/forestproductsdemand
git pull origin master
git add
git commit -m "Explanatory message"
git push origin master
Alternatively "git commit -a"" is a replacement for "git add"" and "git commit". What is the difference between pull and clone: "I like to think of 'clone' as "make me a local copy of that repo" and 'pull' as "get me the updates from some specified remote."

The commands to setup a fresh repository from bitbucket :
mkdir /path/to/your/project
cd /path/to/your/project
git init
git remote add origin ssh://git@bitbucket.org/username/bbreponame.git
# Upload and set the local changes as upstream
git push -u origin master
See also this discussion on why do I need to set upstream?

Commands to copy an existing repository from bitbucket :
 git clone git@bitbucket.org/username/bbreponame.git

Go back in time 

Display the modification log
git log 
Display the log of a particular branch (after a fetch for example)
git log origin/master
Display a compact log for one file or one directory only
git log --abbrev-commit --pretty=oneline path_to_file
Identify the commit identity in the log and copy its sha number. Then to go back to this state for the whole folder: 
git reset --hard commit_sha
To go back to this state for only one file, see git checkout
git checkout commit_hash  path_to_file/file_name
No commit hash to get to get the file back to the latest commit.

Chekout the older revision of a file under a new name

git show commit_sha:filename > new_file_name
See also alias and git grep below.

 Help

Get help on a command (will start a web browser):
git init --help

Configure user name and email

Display your user name, email and remote repositories
git config -l
To change username and email
git config --global user.name "Your Name"
git config --global user.email you@example.com
Setting your email in git explains how to change the email for the current repository only.

Branching

To start work in a new branch:
git branch new_branch_name
git checkout
new_branch_name
To compare a file between 2 branches:
git diff branch1 branch2 file_name
To merge changes back to the master branch:
git checkout master
git merge branch1
If there were conflicts, they will be presented in this way:
"The area where a pair of conflicting changes happened is marked with markers <<<<<<<, =======, and >>>>>>>. The part before the ======= is typically your side, and the part afterwards is typically their side."

I might need to delete a branch at some point:
git branch -d branchname
Delete a remote branch (stackoverflow question)
git push --delete origin temp
Deleting your master branch.

If I am on a detached head, it is recommended to create a temporary branch (stackoverflow).
git branch temp
git checkout temp
git add -a
git  commit -m "description of changes"
git checkout master
git merge temp
Delete uncommitted changes in current working directory:
git checkout branch_name .
See also below git clean.

Add minor change to the previous commit (git commit --amend):
git commit --amend

Tagging

Creating an annotated tag
git tag -a v1.4 -m 'my version 1.4'
You can add a tag after the fact. To tag an earlier commit, specify the commit checksum or part of it:
git log --pretty=oneline
git tag -a v1.2 -m 'version 1.2' 9fceb02
Delete a tag
git tag -d tag_name

A regular push command won't push a tag (bitbucket), to push all your tags :
git push origin --tags

Display changes

To view modified files that have not been committed and to view commit history you can use: 
git status
git log

git log --pretty=oneline
Shows the changes between the working directory and the index.
git diff

Shows the changes between the index and the HEAD
git diff --cached
Shows all the changes between the working directory and HEAD
git diff HEAD
The 3 lines above were copied from this question on git diff.


Show when the tip of branches have been updated
git reflog
 Alternatively, call the repository browser with:
gitk
To view a shorter version of the log file, and get an idea at where I am in the history:
git log --graph --decorate --all --pretty=oneline
You can define an alias for git log as explained be Fred here:
git config --global alias.lg "log --color --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr)%C(bold blue)<%an>%Creset' --abbrev-commit"
The new alias can then be used with
git lg
Use tags to specify important points in history, such as software versions.

Working with files

Get back a file to the last commit 
git checkout path_to_file/file_name
Get back a file to a previous commit, using the commit hash
git checkout 4fb987f175210c09daaa4d0240070ffc9641120b path_to_file/file_name
Rename a file
git mv file_name file_name_new
Change the case of a file on a windows FAT 32 system:
git mv load.r load2.R
git mv load2.R load.R
Sometimes the vi editor starts. To exit the vi editor:
ESC:q!
If a file or folder has been renamed outside of git, I get this warning:
$ git add .
warning: You ran 'git add' with neither '-A (--all)' or '--ignore-remo
whose behaviour will change in Git 2.0 with respect to paths you removed
Paths like 'docs/efi/efi_logo_rgb_small_siw.jpg' that are
removed from your working tree are ignored with this version of Git.

* 'git add --ignore-removal ', which is the current default,
  ignores paths you removed from your working tree.

* 'git add --all ' will let you also record the removals.
Therefore I think I should always run "git add --all "

Remove local (untracked) files from my current Git branch
Show what will be deleted with the -n option:
git clean -f -n
Then - beware: this will delete files - run:
git clean -f
Alternatively clean in interactive mode:
git clean -i

Search text

 Search all files in the subdirectory "subdir" for lines containing the words "factor" and "item". Show 2 lines of context (2 leading and 2 trailing lines).
git grep -e item --and -e factor -C 2 -- subdir/
Stackoverflow:  How to search committed code in the git history?

Bulk replace strings

Use git grep to replace strings in many files in the directory :
git grep -l 'original_text' | xargs sed -i 's/original_text/new_text/g'

Save local modification temporarily

# I had edited the current file in between so needed to use
git stash # save local modifications away
git checkout __commit__hash__
# do some stuff in there ...
# Get back to most recent version of the code
git checkout branch_name
git stash pop # reload local modifications

.gitignore

To ignore all files in a folder but not the folder itself. Put this .gitignore into the folder, then git add .gitignore
*
!.gitignore
 To exclude everything except a specific directory foo/bar (note the /* - without the slash, the wildcard would also exclude everything within foo/bar):
    /*
    !/foo
    /foo/*
    !/foo/bar

Remote

When a repository is connected to several remote repositories, to change the default git remote, push with :
git push -u origin master
Then later push of that branch to that remote can be made simply with:
git push

Another command without specifying the remote and the branch
$ git push -u
fatal: The current branch master has no upstream branch.
To push the current branch and set the remote as upstream, use

    git push --set-upstream origin master

After I run this set upstream flag, I can push to the remote server. Then I get this message

[...]
 * [new branch]      master -> master
Branch master set up to track remote branch master from origin.
I'll have to figure out what this does.

Using the gh-branch to publish project documentation on github

SO Answer to the question "How to add a git repo as a submodule of itself? (Or: How to generate GitHub Pages programmatically?)": An alternative to using Git Submodules to generate GitHub Pages is to use Git Subtree Merge Strategy.

In fact I didn't use quite that strategy and I instead cloned a temporary copy of my repository. Created the gh-page  branch. Pushed it to github. Then I went back to the original repository (where I have a few large untracked data files I find handy to keep for analyses purposes).

Then within the inst folder, I cloned only the gh-branch. To clone only one branch:
git clone -b mybranch --single-branch git://sub.domain.com/repo.git
Then I renamed the folder to "web", so that I had a inst/web folder, tracking the gh-branch. inst/web is ignored in the main repository.

References

Presentations:
Workflow:
Turorial:

Thursday, August 22, 2013

Export Excel files with multiple sheets from R using XLSX

require(xlsx)
wb = createWorkbook()

# Create a first sheet
sheet  = createSheet(wb, sheetName="NewSheet.1")
rows = createRow(sheet, rowIndex=1:2)
cells = createCell(rows, colIndex=1:8)
setCellValue(cells[[1,1]],"Text")
setCellValue(cells[[1,2]],-0.1)
setCellValue(cells[[1,3]],"=B1+1")

# Create another sheet
sheet  = createSheet(wb, sheetName="NewSheet.2")
rows = createRow(sheet, rowIndex=1:2)
cells = createCell(rows, colIndex=1:8)
setCellValue(cells[[1,1]],"Text")

# Save Excel file and show me the names of the sheets that have been created
saveWorkbook(wb, "trying out.xlsx")
names(getSheets(wb) )

XLSX Package on CRAN

XLSX is based on the Apache  POI project. I still need to figure out  how to evaluate formulas in the Excel file. There may be some help at the Apache POI formula evaluation page.

Wednesday, February 23, 2011

if (window.focus) window.focus()

Object detection The whole point is that if you want to use the array document.images, first check if it is supported. If you want to use the focus method of the window, first check if it is supported.

If you always use object detection, your scripts will never generate any error messages, although they might not work in certain browsers.

Tuesday, November 16, 2010

Chararcter set detection python script

Charset detection script exctracted from the mozilla detection script: chardet
I tried it and it works.

Thursday, October 07, 2010

Nice way to insert a stylesheet in an HTML doc

I find this is a nice way to insert a stylesheet in a HTML document. Just specify it in the <HEAD> section, between <STYLE> mark-ups.
like this:


<STYLE>
<!--
BODY,DIV,TABLE,THEAD,TBODY,TFOOT,TR,TH,TD,P { font-family:"Arial"; font-size:x-small }
-->
</STYLE>


I used this tool to convert the html code in HTML entities.

Tuesday, September 07, 2010

Virtual Python

What It Does

virtualenv is a tool to create isolated Python environments.

The basic problem being addressed is one of dependencies and versions, and indirectly permissions. Imagine you have an application that needs version 1 of LibFoo, but another application requires version 2. How can you use both these applications? If you install everything into /usr/lib/python2.4/site-packages (or whatever your platform's standard location is), it's easy to end up in a situation where you unintentionally upgrade an application that shouldn't be upgraded.

Or more generally, what if you want to install an application and leave it be? If an application works, any change in its libraries or the versions of those libraries can break the application.

Also, what if you can't install packages into the global site-packages directory? For instance, on a shared host.

In all these cases, virtualenv can help you. It creates an environment that has its own installation directories, that doesn't share libraries with other virtualenv environments (and optionally doesn't use the globally installed libraries either).

Friday, August 06, 2010

Python CGI scripts under wamp

If you want to run cgi script using wamp server on windows, you need to edit the Apache configuration file httpd.conf. Use the command line as administrator and run
C:\wamp\bin\apache\Apache2.2.11\conf>notepad httpd.conf
Then search for the line containing "addhandler" and write:
AddHandler cgi-script .py
For python scripts (for any other scripting language, just use the relevant extension).
Then the first line of your script should contain the path to the python program:
#!c:/Python26/python.exe -u
According to python cgi on windows.
Place your script in C:\wamp\bin\apache\Apache2.2.11\cgi-bin\myscript.py and you're done.
The output of the script can be accessed at
http://localhost/cgi-bin/myscript.py

The script works but there are restrictions on the modules I can import. I can import native python modules, but not the module I made myself in other places outside the cgi-bin folder. I don't know how to solve that yet. (This is probably a security restriction)

pyodbcs connection strings

Pyodbc conncetion strings gives a useful piece of advice concerning ODBC connection to Excel.
The Excel driver does not support transactions, so you will need to use the autocommit keyword:
cnxn = pyodbc.connect('DSN=test', autocommit=True)

DSN (Data Source Names) are defined in the ODBC data source administrator of the control pannel in windows (in the section administrative tools).

I wasn't able to establish a DSN-less connection.

Tuesday, August 03, 2010

Link to a specific part of a YouTube video

Matt cuts blog
If you want to link to a specific part of a video on YouTube, you can. For example, http://www.youtube.com/watch?v=PjDw3azfZWI#t=31m08s
Notice the “#t=31m08s” on the end of the url? That link will take you 31 minutes and 8 seconds into that video.

Monday, August 02, 2010

Write tests in programming

In this lecture, Richard Buckland says that tests should be written before the functions.