Setup of programs I use daily

Research Software

• LyX to write articles and structured documents including citations, Check my blog post on LyX
• JabRef to manage a bibliography
• LaTeX to typeset publications, exported in PDF format
• R statistical software and the R-Studio application
• Several packages: plyr, ggplot2, FAOSTAT, testthat
• Knitr and R Markdown to produce work reports in html. Knitr and LaTeX to produce more elaborate reports in PDF
• xlsx to write data frames to Excel

OS

• Debian GNU-Linux at work
• Ubuntu and Lubuntu at home
• Until 2014 windows XP and windows 7 at work 

Office Software

• I use Firefox (renamed iceweasel on Debian) for web browsing, with the following extensions:
• Adblock to block annoying ads
• Delicious bookmarks to have mobile bookmarks across many browsers
• Mindthetime to keep track of time usage. This extension is not working any more.
• For simple image editing I use Irfanview. I crop scanned pictures or screenshots and I redimension them for web use or use in documents. Very occasionally, I use the "sharpen" function or the brightness, contrast, gamma corrections. 
• I use Microsoft word and Excel at work. I use Google documents when writing something at home.
• To work collaboratively I use media wiki with the extension Semantic Mediawiki
• Based in this review, I installed spacesniffer it helps analyse disk space usage on windows. Like Disk space analyser on GNU-Linux systems. You can use filters such as this one: "*.jpg;>100kb;>2weeks" to filter only certain type of files, sizes and dates.

Command line tools on GNU-Linux

• du to analyse disk usage. I created an alias duh for du -h.
• grep to search inside text files
• pdfgrep to search inside pdf documents
• git see programming below

Thesis writting

• Writing documents in Lyx
• Keeping track of versions with git
• Managing a bibliography with Jabref.

 Programming software

• Scite to edit code
• Git to track versions of code. See also git commands and posts tagged with git.

I want to test

• hubic remote for git annex
• Etherpad (within a company) 
• Kaban (within a company)
• Sage mathematics software 
• Panda data analysis because it brings data frame structures to python
• Montage Video sur ubuntu and Ubuntu video tools

 I stopped using

• QuickWiki Quick search of definitions
• Zotero to share search results with colleagues
• Mendeley to insert citations in Microsoft Word and to manage a bibliography (can import bibtex files form Jabref) 

Links

• On this blog, linux users describe their setup
• usethis.com another site where users such as Yihui Xie describe their setup.

Singularity - the opinion of Peter Norving

Interview of Peter Norving: "Do you agree with Kurzweil's forecast regarding the singularity?"

"So Ray is great and I appreciate all that he's done, I appreciate all the advances in AI that he has done in reading, in music and so on, so he's a great technologist. I think in terms of these predictions, euh, I think he's doing the best he can by saying, I wanna nail down this prediction, I'm gonna look at the data that we have, I'm gonna extrapolate from that data and come up with an answer, euh, I think the only problem is that the data is not conclusive, right. So you look at, Moore's law, yes, you can say computers are going at this particular rate, but I don't think you can necessarily say that when we pass a certain number of transistors, that corresponds to a certain number of neurons and therefore the 2 are equivalent, I think that's where the analogy breaks down."

What are your thoughts on molecular manufacturing and Bio-Engineering?

"So it's fascinating to me, euh, I'm trying to learn something about it. I'm educating myself, I gotta long way to go. I think it does tie-in because it is a mix between the physical world and the world of information. Euh, I think it is likely that in the coming century a large part of manufacturing will be done with cells or with atoms. I kind of think it'll be more done with cells than with atoms just because they're bigger, they're easy to manipulate, they've already got this 3 Giga bite computer on board, we just gotta figure out how to program it."

Peter Norving is director of research at google.

Home server

This company is selling a home server. For what it's worth! ;-)
They are based in Anger, how bizarre! And their website is in French and German.

Microsoft's open source license

Microsoft's open source license for Non-Commercial Academic Use Only: "You may create derivative works of the Software source code and distribute the modified Software solely for non-commercial academic purposes, as provided herein.. If you distribute the Software or any derivative works of the Software, you will distribute them under the same terms and conditions as in this license, and you will not grant other rights to the Software or derivative works that are different from those provided by this MSR-LA. Your license rights to the Software (or any Microsoft intellectual property associated therewith) does not include any license, right, power or authority to subject the Software or derivative works thereof in whole or in part to the terms of any license that requires as a condition of use, modification and/or distribution of software subject to such license that the software or other software combined and/or distributed with such software be (A) disclosed or distributed in source code form; (B) licensed for the purpose of making derivative works; or (C) redistributable at no charge."
Bla bla bla. :-) You're allowed to modify it but you're not allowed to licensie it under another license that would allow the distribution in source code form or distribution at no charge.

Python, Delphi and oracle

"In Greek mythology Python, serpent, was the earth-dragon of Delphi, always represented in Greek sculpture and vase-paintings as a serpent. She^[1] presided at the Delphic oracle, which existed in the cult center for her mother, Gaia, Earth, Pytho being the place name that was substituted for the earlier Krisa.^[2] Hellenes considered the site to be the center of the earth, represented by a stone, the omphalos or navel, which Python guarded."

From the wikipedia article Python (pythology) (nice typo! :).

In computer fields
python is a programmin language and Delphi too. Oracle is a software company and you wonder where their inspiration came from.

Fibre optique jusqu'au point d'attache Free

http://ftth.free.fr/contexte_technique.html

US department of defence advocates Open Source

US Department of Defence: "To effectively achieve its missions, the Department of Defense must develop and update its software-based capabilities faster than ever, to anticipate new threats and respond to continuously changing requirements. The use of Open Source Software (OSS) can provide advantages in this regard." (pdf on the website of David A. Wheeler)

Basic web server on a microcontroler

Web Xweb32 Lite is a web server that runs on a microcontroler ATMEGA32. With that you can visualise variables within a webbrowser. This is even smaller than any barebone computer.

A PC in the keyboard

A PC that fits in the keyboard for 99 EUR with linux: Northteck Gecko Surboard (article on Linux Devices.com)
Xcore is the manufacturer.

Géolocalisation par wifi

Le service de géolocalisation est assuré par la technologie Wi-Fi Positioning System (WPS). Grâce au Wi-Fi intégré, les cartes Eye-Fi détectent les réseaux Wi-Fi environnants pendant que vous prenez des photos. Une fois que les photos ont été transférées, le service Eye-Fi leur associe des données géographiques.

Connect to a SQLite database using python

SQLite is included since python 2.5 I connected to a SQLite database created from zotero that way:

import sqlite3 as sqlite
con = sqlite.connect('zotero.sqlite')
cur = con.cursor()
cur.execute('CREATE TABLE foo (o_id INTEGER PRIMARY KEY, fruit VARCHAR(20), veges VARCHAR(30))')
con.commit()
cur.execute('INSERT INTO foo (o_id, fruit, veges) VALUES(NULL, "apple", "broccoli")')
con.commit()
print cur.lastrowid

cur.execute('SELECT * FROM foo')
print cur.fetchall()

Here is the output:
>pythonw -u "test_sqlite.py"
1
[(1, u'apple', u'broccoli')]
2

With help from DZone snippets and devshed. However devsched's information about downloading and building the sqlite library is outdated as it is now included in python.


Edit:
In a later post, I explain how to connect do an SQLite database with the R statistical software and a package called dplyr.

Visualise the history of python development

In this blog I found a link to this video of the history of python development from the release date in 1991 to the time when Gido Van Rossum was hired by google in dec. 2005.

First plot with python (x,y)

Showing a plot with pylab module from the python (x,y) package that I just installed on windows.

from pylab import *
test_list = [1,2,6,123,51,32]
plot(test_list)
title('Test plot of a list of numbers')
show()

Metaprogramming

Metaprogramming " is the writing of computer programs that write or manipulate other programs (or themselves) as their data."

Python at google, quotes

In February 2006 Greg Stein spoke about Python at google. I liked this quote: "Greg mentioned that to create code.google.com took about 100 lines of python code. But since it has so many dependencies, the build system generated a 3 megabyte makefile for it!" That gave me an idea of what a *glue* language means!

On the same blog there is another quote from Guido Van Rossum: "Guido uses print statements for 90% of his debugging and a debugger for the times when his brain stops working or he has to debug some wierd code that someone else wrote. "

Optical mouse emitting beams of light.

This is my optical mouse at work:

Compress images in a microsoft word file

When you have a word file over 3Mb, it might help to reduce the size of the photos. The problem is often that you use a resolution too high for the print quality.
Let's double click on one of the picture you inserted in your word document, chose [compress] and click on [all pictures in document]. You can now change the resolution to a print quality. This can reduce your file size to a half or a third if you have inserted very big photos from a 7Mb digicam fro example.

Here is a screenshot from the action:




Enjoy the small file size!

use a batch script from cygwin

Yes you can start a batch script from cygwin. What did I use this for? Well, it's usefull to restart a windows service for example with the DOS command
net stop service_name
net start service_name
You can then login remotely with ssh in your cygwin shell and restart the windows service or start any command you can start from dos, just write the commands in a .batch script.

nevow ,the templating engine from twisted needs Zope.

I installed nevow, the templating engine from twisted to create web pages on windows. The problem was that I had to install zope for windows as well. I don't now why.
I plan to use this templating engine because my web page generating scripts are very ugly, with plenty of if, elif, elif and one cannot find the structure of a web page in there.

xls2csv, using python win32com and excel

This python script change a bunch of excel files in cvs files, needs Excel installed on the computer.

#
# Save all Excel files in the current dir as csv files
# (CSV = comma separeted values)

from win32com.client import Dispatch
import os,time

xlCSV = 6 #a vba constant for this fileformat
xlApp = Dispatch("Excel.Application")
xlApp.Visible = 1

def saveAsCSV(filename):
    xlApp.Workbooks.Open(filename)
    xlApp.ActiveWorkbook.SaveAs(filename[:-3]+"csv",xlCSV)
    xlApp.ActiveWorkbook.Close(SaveChanges=0)

for root, dirs, files in os.walk(''):
 for file in files:
  if ".xls" in file:
                    print file
                    saveAsCSV(os.path.abspath(file))
xlApp.Quit()
del xlApp

If you need to do this on linux, you can try that program: xls2csv.