A year on Debian

Inspired by other users seen on the Linux setup. I switched my work computer to Debian GNU/Linux in January 2015. I used the Jessie version which was "unstable" at the time. It became "stable" in april 2015 and I think the adjective is deserved. I'm happy.


I mostly use the R statistical software, the iceweasel web browser, the Lyx editor, the reference manager Jabref and the Evolution mail and calendar program. I learned a lot by using git (file revision system) and databases through the command line. I sometimes use vim to edit text files and I use the RStudio editor in Vim mode. I also use the Libre office Calc, Writer and Draw software.

As long as you stick with what is in the huge Debian package repository, software updates are easy. If you need programs updated very recently (in the last months), installation can become tricky. Although it mostly involves adding new software repositories or installing .deb files directly. For example I managed to install the proprietary statistical software STATA.

Using SSH keys to access remote servers and git repositories

An SSH key can be used to access a virtual private server or a remote git repository without the need to enter a password every time. By sharing your public key with the remote server, your compter is authenticated as a trusted access point.

Creating SSH keys 

In Debian GNU Linux, using the Gnome desktop, you can create a private and public SSH key pair with for example the seahorse key manager. Under File / New / Secure Shell Key.

Created keys will be visible under ~/.ssh/ the private key is called id_rsa and the public key id_rsa.pub. You should only share the public key.

At the command line, you can create keys with

ssh-keygen -t rsa -C "your_email@example.com"

Virtual Private Server

I bought a virtual private server with Debian pre-installed. A public key can be added in the file ~/.ssh/authorized_keys. When connected to the server, edit the file:

vim ~/.ssh/authorized_keys

You might need to change access permission to that file as explained in this gist.

Bitbucket

Your public key can be added to your bitbucket account under manage account / security / SSH keys. This page explains how to use the SSH protocol with Bitbucket in more details.

Github

Your public key can be added to your Github account under profile / settings / SSH key. More details on how to generate and use SSH keys for github.

Then at the top of your Github repository you should see the "clone URL". Copy the SSH URL, in the form: git@github.com:yourusername/yourrepository.git
Add it as a remote origin:

git remote add origin git@github.com:yourusername/yourrepository.git

If there was already a remote repository you might need to delete it first with git remote remove origin.



The push and set the remote repository as an upstream repository:

git push --set-upstream origin master

Subsequent push can be simply made with

git push

See also

See also my other blog posts on the bash shell commands and on git commands.

Ipython notebook and R

I chose to use python 3. Several of the shell commands below have a "3" suffix in Debian testing as of April 2015: ipython3, pip3.

Install programs

I installed ipython-3-notebook (in Debian Jessie) from the synaptic package manager.

In order to install the R module, I installed PIP for python 3 in the synaptic package manager. PIP is the Python Package Index, a module installation tool. Then I used pip3 to install rpy2

sudo pip3 install rpy2

There is a blog post on how to avoid using sudo to install pip modules.

Install statsmodel, a module for statistical modelling and econometrics in python. Maybe I should have installed python-statsmodels as a Debian package instead? But I it seems to be linked to python 2.x instead of python 3 (it had a dependency on python 2.7-dev). Therefore I installed statsmodels with pip3, using the --user flag mentioned above to install is as a user only module.

pip3 install --user statsmodels

The installation took several minutes on my system. It seemed to be installing a number of dependencies. Many warnings about variables defined but not used were returned but the installation kept running. The final message was:

Successfully installed statsmodels numpy scipy pandas patsy python-dateutil pytz

Cleaning up...

Starting the Ipython notebook

Move to a directory where the notebooks will be stored, start a ipython notebook kernel

cd python
ipython3 notebook

Shortcuts

See also the Ipython Notebook shortcuts. Useful shorcuts are ESCAPE to go in navigation mode, ENTER, to enter edit mode. It seems one can use vim navigation keys j and k to move up and down cells. Pressing the "d" key twice deletes a cell. CTRL+ENTER run cell in place, SHIFT+ENTER to run the cell and jump to the next one, and ALT+ENTER to run the cell and insert a new cell below. 

Run R commands in the Ipython notebook

Load an ipython extension that deals with R commands

%load_ext rpy2.ipython

 Display a standard R dataset

%R head(cars)
%R plot(cars)

Use data from the python statsmodels module based on this page.

import statsmodels.datasets as sd
data = sd.longley.load_pandas()

Print column names of the dataset

print(data.endog_name)
print(data.exog_name)

Print a dataset as an html table by simply giving its name in the cell. For example this data frame contains exogenous variables:

data.exog

Python can pass variables to R with the following command:

totemp = data.endog
gnp = data.exog['GNP']
%R -i totemp,gnp

Estimate a linear model with R

%%R
fit <- br="" gnp="" least-squares="" lm="" nbsp="" regression="" totemp="">print(fit$coefficients)  # Display the coefficients of the fit.
plot(gnp, totemp)  # Plot the data points.
abline(fit)  # And plot the linear regression.

Plot the datapoints and linear regression with the ggplot2 package

%%R
library(ggplot2)
ggplot(data = NULL, aes(x =gnp, y = totemp)) +
    geom_point() +
    geom_abline( aes(intercept=coef(fit)[1], slope=coef(fit)[2]))

Virtual Machine setup for development purposes

Creating a Virtual machine with Vagrant and PuPHeT.


According to those 2013 stack overflow questions, there were many reasons not to develop in a VM, unless one had to specifically develop for several OS:

• Thoughts on Development using Virtual Machines 
• Should a development machine be inside a VM?

But in the same year, the PhPHet developer explained why he thinks that one has to develop in a virtual machine.

Running a VM 

I followed the vagrant instructions to install a basic VM.

vagrant init hashicorp/precise32 vagrant up

"The guest machine entered an invalid state while waiting for it
to boot. " [...] "If the provider you're using has a GUI that comes with it, it is often helpful to open that and watch the machine"
I started the virtual machine in virtual box, an error message came up: 

"VT-x is disabled in the BIOS. (VERR_VMX_MSR_VMXON_DISABLED)."

Under Machine / Settings/ System / Acceleration, I disabled the Hardware virtualisation. The VM could then start. This works for 32 bits systems. Unfortunately 64 bit systems require hardware virtualisation, this means I cannot change this setting for 64 systems. I'll have to enable VT-x in the BIOS later on.

After I installed Virtual box, my mouse was rendered invisible. This may be due to the fact that the mouse was captured and that I didn't know the host capture key (default to the right Ctrl key) to free the mouse from the virtual machine's window.

Connecting to the virtual machine

Connecting from the virtual box GUI. The default user is "vagrant" and password "vagrant".

Connecting with SSH into the machine from a command prompt:

 vagrant ssh

 

Shared folder

A folder can be share with the host operating system. In virtual Box settings for the machine, under shared folder, create a machine folder and set it to auto-mount in the guest operating system.

Other tools

• A vagrant tutorial by bertvv mentioned another tool called packer.

• Installation de Docker sur un serveur dédié Kimsufi OVH sous Debian et chargement d'un conteneur

Messages by the vagrant creator

Tao of hashicorp
Comparing Filesystem Performance in Virtual Machines Automation Obsessed

Octave commands

I am trying to run Matlab based test statistics in GNU Octave.

Octave commands 

List variables available in memory

who %this is a comment
whos %provides class details

Change and display working directory

cd directory_name
pwd

Manipulate data structures:

x.a = 1;
x.b = [1, 2; 3, 4];
x.c = "string";

Display the value of a variable

disp(x)

Loop over a list of files

csvfiles = dir("*.csv")
for file= csvfiles'
fprintf(1,'Doing something with %s\n',file.name)
end

Creating character arrays

"In the MATLAB^® computing environment, all variables are arrays, and strings are of type char (character arrays)."

Reading data from an Excel or CSV file

The test statistics I wanted to use loads data from an Excel file but this returned the error :" 'xlsread' undefined ". Reading excel file is provided by the IO package which is not installed by default. The package is available in the Debian repository under "octave-io" , with the description "This package [...] contains functions to [...] read Excel spreadsheet (xlsread) and OpenDocument spreadsheet (odsread)." It is based on Apache POI. Load the package an try to read a file:

pkg load io;
data=xlsread('file_name.xls');

xlsread returns an error "Detected XLS interfaces: None."  This forum post recommends to load the java and windows packages as well. Those packages are not available in the Debian repositories.
I decided to convert the Excel file to csv and use csvread instead.

The script now gives the same output as on a windows machine running Matlab.

Warning: possible Matlab-style short-circuit operator 

Short-circuit boolean operators explains that:

"MATLAB has special behavior that allows the operators ‘&’ and ‘|’ to short-circuit when used in the truth expression for if and while statements. The Octave parser may be instructed to behave in the same manner, but its use is strongly discouraged." [...]

I wonder why it is strongly discouraged. 

"To obtain short-circuit behavior for logical expressions in new programs, you should always use the ‘&&’ and ‘||’ operators."

I replaced "|" by "||" in the code.

Writing test results to a file

Matlab low level file IO, explains how to use fprintf (a vectorised implementation of the c function) to write text data to a file.

Why should research organisations release free software?

Research organisations protect software with Intellectual Property (IP) rights. Some of these IP rights authorise the release of source code but some prevent source code release. Within the organisation, a decision maker should ask herself:

• Can the organisation pay a person or a group of person in years to come to maintain that program in the long run?

If the answer is no, read on.

Researchers frequently move to other job positions. Once a researcher has moved to another job, the software codes he/she wrote is likely to sit idle on the organisation's storage drives. When no insider knows how to modify a computer program's code, the value of that program for the organisation will depend on the possibility for outsiders to modify the code.

1. If researchers outside the organisation are not allowed to update the software, it will not be used. IP rights preventing source code modification don't have any value.
2. If on the other hand, the piece of software is released as free and open source software, researchers outside the organisations are likely to update the software once the need arises. IP rights ensure that the first creator's contribution with its organisation's affiliation remains cast in the software's stone. An acknowledgement mentioning the organisation will travel with the piece of software as long as this piece of code is useful. This is likely to attract future project contribution and funding to the host organisation.

Installing STATA on Debian GNU-LINUX

I needed to install STATA to collaborate with a colleague at work. The computer guy gave me the software on a disk, with an installation guide. Here are the commands I entered following those instructions:

Create a directory for Stata

# mkdir /usr/local/stata13
# ln -s /usr/local/stata13/ /usr/local/stata

Install Stata

# cd /usr/local/stata13
# /media/paul/Stata/install

Stata 13 installation
---------------------

  1.  uncompressing files
  2.  extracting files
  3.  setting permissions

Done.  The next step is to run the license installer.  Type:

        ./stinit

If the licensed software is Stata/IC 13, you will be able to run Stata/IC by typing
        xstata              (Run windowed version of Stata/IC)
        stata               (Run console  version of Stata/IC)

Run the license installer

./stinit

There follows some questions about user name and affiliation. "The two lines, jointly, should not be longer than 67 characters."
Then comes the message:

Stata is initialized.
You should now, as superuser, verify that you can enter Stata by typing

        # ./stata
or
    # ./xstata

I added this to my .bashrc so that stata and xstata can be used as a command directly:

 export PATH=$PATH:/usr/local/stata

Both command "stata" and "xstata" work as a normal user now.

There is an error message when running xstata:

'Failed to load module "canberra-gtk-module"'

But this was not a problem at the start.

GNOME application launcher

I added STATA to the GNOME application lancher, by typing "application" in the launcher, then "main menu", "new menu".

R to Stata

I use R most of the time for data analysis and will export csv files to STATA.
R command to export csv files:

write.csv(dtf, "filename.csv", row.names = FALSE, na = ".")

STATA command to import csv files:

insheet using "filename.csv", delimiter(",")

Bulk resize images with a shell script and imagemagick

I like to resize images to 1000 pixels wide before sending them per email, it's large enough for most screens and I can send a dozen of pictures without cluttering my friends inbox. A pity that in 2015 there is still no easy way to do this integrated in the Gnome file manager. Maybe there is another way, or maybe I should use a proper image management program. Add a comment on how you resize images. 
 
But it's easy to create a shell script:

#!/bin/sh
mkdir -p small
for i in *.jpg; do
    echo $i;
    convert $i -resize 1000x small/$(basename $i .jpg).jpg;
done

Since I had images ending with .JPG extension in capital letters, I added a second loop:

for i in *.JPG; do
    echo $i;
    convert $i -resize 1000x small/$(basename $i .JPG).jpg;
done

Inspired by this question on stack overflow, and the mkdir complain corrected by looking at this question.

Debian dist-upgrade from Wheezy to Jessie

I needed a couple of recent software versions (Lyx version> 2.1 and Latex), and they were not available in the wheezy-backports, therefore I decided to upgrade from Debian Wheezy, the current stable version to Debian Jessie, the testing version. Debian Install FAQ.

This page gives the configuration changes and commands to be used:

1. Edit /etc/apt/sources.list and replace all occurrences of "wheezy" with "jessie".
2. run apt-get update
3. run apt-get upgrade
4. run apt-get dist-upgrade

For several reasons the upgrade process didn't complete properly. After a system reboot, there was no networking, no graphical desktop and even after starting the Gnome desktop by hand with "startx", there were several issues with system settings, such as printer setup.

Networking

I had an issue with networking not working.
According to the debian page on network setup, changes below are not recommended (see below).

I fixed by editing/etc/network/interfaces

# Lines added from odoepner.wordpress.com
auto eth0 

allow-hotplug eth0

iface eth0 inet dhcp

Then ran as root

etc/init.d/networking start

And network was working again.

This page explains how to put this network as a managed interface again: 

But I shouldn't have done this.
According to the debian page on network setup: "Keep configuration of "/etc/network/interfaces" as simple as in the following".
auto lo
iface lo inet loopback

No desktop manager at startup

I realised that there was no desktop manager at startup. I first followed answers to this question and edited  /etc/inittab. But it is not needed, because the boot manager systemd doesn't look into that configuration file.
I spent some time reading about a systemd controversy in Debian and why it doesn't matter so much in the end.

For the moment I start the Gnome desktop from a terminal with the command:

xstart

It looks like not all package had been upgraded.
I ran as root

apt-get upgrade -f

The following packages will be upgraded:
  live-tools

[...]

Preparing to unpack .../live-tools_4.0.2-1_all.deb ...
dpkg-divert: error: rename involves overwriting `/usr/bin/uptime' with
  different file `/usr/bin/uptime.orig.procps', not allowed

From Synaptic, I did a complete removal of the package live-tools.
After that many configuration steps took place. Printer setup utility was working again. And now there only remains a bluetooth issue.

Bluetooth issue

dpkg: error processing package gnome-bluetooth (--configure):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 bluez
 bluetooth
 gnome-bluetooth

Lyx upgraded to version 2.1.2

At least Lyx was upgraded to version 2.1.2. I had to "reconfigure LyX with Tools→Reconfigure;".Then I could create documents with Lyx again. And the R integration worked as well.

Upgrade might not have been needed

Afterwards, I realised that an upgrade might not have been needed:

"If you want to install a single package in Debian, you do not need to update the whole system. It can be done with three commands by inserting the repos for Testing and Unstable in /etc/apt/sources.list, fixing the distro you (mostly) wish by setting APT::Default-Release "stable" in /etc/apt/apt.conf.d/local, then doing aptitude install packagae_name/testing -t testing"

GNU screen for long running server processes

Use screen to keep a long process running on a server after you close the ssh session. I started a screen session with:

    screen -S sessionname

In order to find the screen session later you might want to rename it using sessionname. Or on the first screen invocation use the s flag -S sessionname
I started the R software in this screen session, started a long running process. Then detached the session with:

    CTRL-A-D

I could re-attach the session later with:

    screen -r sessionname

If the session was not detached properly, it might be necessary to detach it and re attach it:

   screen -d -r sessionname

Screen detach before leaving the ssh session. You may want to use the autodetach configuration option.

Log out of your ssh session. Log in back again later:

screen -ls # list sessions
screen -r sessionname # attach a session

There might be solutions to move an already started process to GNU screen but its not straightforward:

"You cannot do this, easily. I'd suggest making it a habit to start screen as the first thing you do after opening a console. However, for your actual problem, there's another thing you could try: after having launched your job from the terminal, background it by typing ctrl-z and then bg. After that, detach the job from it's parent shell; in bash you'd do disown -h %. After that, you can safely close the terminal and the job will continue running."

More tips in this discussion on screen.

SSH tunnel and port forwarding

• Ubuntu Remote_Port_Forwarding

• ssh -C -D 1080 laptop

• http://straightedgelinux.com/blog/howto/socks.html
• ssh -N -D 1080 klaatu@home.linuxserver.com 
• https://wiki.debian.org/SOCKS
•  Assuming your SOCKS server is running on your localhost, listening on port 20000, you could run one of the following commands:
• ssh root@mysshserver -o ProxyCommand='connect-proxy -S localhost:20000 %h %p'
• http://www.debian-administration.org/article/449/SSH_dynamic_port_forwarding_with_SOCKS

• ssh -D 1080 shell.example.org
  tsocks thunderbird

   

  Ubuntu Remote_Port_Forwarding

        ssh -C -D 1080 laptop

    http://straightedgelinux.com/blog/howto/socks.html
        ssh -N -D 1080 klaatu@home.linuxserver.com
    https://wiki.debian.org/SOCKS


http://www.debian-administration.org/article/449/SSH_dynamic_port_forwarding_with_SOCKS

        ssh -D 1080 shell.example.org
        tsocks thunderbird

R, packages and Rstudio install on Debian wheezy

See also my previous post on Debian GNU-Linux installation on a Lenovo T400.

R install

I used the Synaptic package manager to add the R repository for Debian from a nearby mirror, under : settings / repositories / other software / add.
Add this APT line:

deb http://cran.univ-paris1.fr/bin/linux/debian/ wheezy-cran3/

There was an error:

W: GPG error: http://cran.univ-paris1.fr wheezy-cran3/ Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 06F90DE5381BA480

After looking at several forums, and this stackoverflow question, I installed debian-keyring and added the key with the commands:

gpg --keyserver pgpkeys.mit.edu --recv-key 06F90DE5381BA480
gpg -a --export 06F90DE5381BA480 |sudo apt-key add -

I could then install R version 3 from the synaptic package manager.

Rstudio

I downloaded R-studio and installed it. There was a missing dependency for libjpeg62. I installed that package from Synaptic. Then ran the dpkg command to install rstudio.

dpkg -i rstudio-0.98.507-i386.deb

Tools

Then I installed Git in order to clone my R project from an online repository.

git clone  project_repository_url

Packages

Within Rstudio, I installed a few packages:

install.packages(c("plyr", "reshape2", "ggplot2"))
install.packages(c("xtable", "markdown", "devtools"))

devtools

The devtools packages requires a libcurl dev Debian package. You can install it at the shell prompt:

$ sudo apt-get install libcurl4-gnutls-dev

Back at the R prompt

install.packages("devtools")

Other dependencies might be needed, the RStudio page on devtools recommends installing the Debian package r-base-dev.

dplyr

The dplyr package required the latest version of a Rcpp package. Which was not available on my CRAN mirror. I installed it from source, (based on this message):

install.packages("Rcpp", type = "source")

install.packages("dplyr")

xlsx

The xlsx package installation complained:

configure: error: Cannot compile a simple JNI program. See config.log for details.
Make sure you have Java Development Kit installed and correctly registered in R.
If in doubt, re-run "R CMD javareconf" as root.

Required the latest version of java 7. (inspired by this post). I installed openjdk-7 from the synaptic package manager. Then ran

update-alternatives --config java  # Choose java 7 as the default

R CMD javareconf

Then

install.packages("xlsx") # worked

RMySQL

MySQL client and server are installed on my system.
While installing RMySQL, I struggled with a configuration error:

  could not find the MySQL installation include and/or library
  directories.  Manually specify the location of the MySQL
  libraries and the header files and re-run R CMD INSTALL.

This post has an answer (thanks!):

sudo apt-get install libdbd-mysql  libmysqlclient-dev

That fixes the issue!
I can connect to the database

library(RMySQL)
mychannel <- br="" dbconnect="" host="localhost" user="paul" ysql="">                       password="***", dbname="dbname")

R packages which are better installed from the Debian package manager

Some packages, such as ‘minqa’, ‘SparseM’ and ‘car’ return an error when one tries to install them from the R prompt. The can only be installed from the Debian package manager, where they have names starting with "r-cran": "r-cran-car", "r-cran-sparsem", "r-cran-minqa".

Ready to work!

Make word, pdf and html documents with markdown and pandoc

Markdown is a simple text markup language.
Pandoc is a document converter. Pandoc demo and sample command.

Pandoc commands

Convert a markdown file to PDF :

pandoc -o README.pdf README.md

The pandoc man page says: "If  the input or output format is not specified explicitly, pandoc will attempt to guess it from the extensions of the input and output filenames." That's what happens above. However "The input format can be specified using the -r/--read or -f/--from options, the output format using the -w/--write or -t/--to options."

Makefile 

This phsychologist blogs about using a makefile to create beamer presentations.
This researchers providers a make file for pandoc templates.

With this simple make file, I can create Microsoft Word, HTML and PDF documents from the same markdown file:

all: docx pdf html

docx: file.md
        pandoc -o file.docx file.md

pdf: file.md
        pandoc -o file.pdf file.md

html: deliverable.md
        pandoc -o file.html file.md

clean:
        rm -f *.html *.pdf *.docx

To create all documents type

make

To create only a docx type

make docx

To delete all created document type

make clean

Improved makefile with variable

file.pdf : file.md
    pandoc -o file.pdf file.md

%.pdf: %.md
    pandoc -o $@ $<

Guide makefiles:

"Here, we have used the percent (%) character to denote that part of the target and dependency that matches whatever the pattern is used for, and the $< is a special variable (imaging it like $(<)) that means "whatever the depencies are". Another useful variable is $@, which means "the target"."

## Makefile to generate documents based on markdown files
## Inspired by this makefile
## https://github.com/kjhealy/pandoc-templates/blob/master/examples/Makefile
##
## I should use vraibles for filenames
## Command line to converts:

## How to make this using variables?
## No space allowed in file names there could be a replacement but I didn't try
## http://www.cmcrossroads.com/article/gnu-make-meets-file-names-spaces-them

## Markdown extension (e.g. md, markdown, mdown).
MEXT = md
## All markdown files in the working directory
SRC = $(wildcard *.$(MEXT))


DOCX=$(SRC:.md=.docx)
PDFS=$(SRC:.md=.pdf)
HTML=$(SRC:.md=.html)


all: $(PDFS)  $(DOCX)
pdf:    clean $(PDFS)
docx:   clean $(DOCX)
#html:   clean $(HTML)


#scrap : scrap.md
#    pandoc -o scrap.pdf scrap.md


# Separator for these lines need to start with a hard tab, not 4 spaces!
%.pdf: %.md
    pandoc -o $@ $<

%.docx: %.md
    pandoc -o $@ $<

clean:
    rm -f *.html *.pdf *.docx

Debian GNU/Linux

Configuring the system 

Disk partitioning

Recommended Partitioning Scheme: "For new users, personal Debian boxes, home systems, and other single-user setups, a single / partition (plus swap) is probably the easiest, simplest way to go."  [...] " For multi-user systems or systems with lots of disk space, it's best to put /usr, /var, /tmp, and /home each on their own partitions separate from the / partition."

Keyboard layout

I use a laptop with a Finnish keyboard and a docking station with a French keyboard. I chose the "SHIFT+ caps lock" keys to change from one keyboard layout to the other. By default the French keyboad has a comma as decimal separator on the numeric keypad. I changed this under:

 system settings / region and language / layouts / French / Options / Numeric keypad delete key behaviour / 4 level key with dot

Installing programs

Package management with apt-get or aptitude. Aptitude is recommended. Here is a comparison in a forum.

Programming

Install a version tracking system

sudo aptitude install git

Add coloration:

git config --global color.ui true

Install the vim text editor

sudo aptitude install vim

Edit the vim configuration file:

vim ~/.vimrc

In this file add:

syntax on
set tabstop=4
set expandtab
set softtabstop=4
set shiftwidth=4
filetype indent on

au BufNewFile,BufRead *.md set filetype=txt

Non-Free (unfortunately)

Skype on Debian wiki (for 64 architecture) try alternatives Ekiga and Linphone (Linphone use with French ISP free). Load skype for Debian from the website, then:

sudo dpkg --add-architecture i386
sudo apt-get update 

sudo dpkg -i skype-install_file_you_downloaded.deb 
sudo apt-get -f install

Adobe Flash

sudo apt-get install flashplugin-nonfree
sudo update-flashplugin-nonfree --install

Super user

As a super user install the program called "sudo" :

apt-get install sudo

how to-properly-configure-sudoers-file-on-debian-wheezy
I am in the sudo group

adduser paul sudo
  The user `paul' is already a member of `sudo'.

addgroup sudo
  addgroup: The group `sudo' already exists.

But I get the  warning message:

paul is not in the sudoers file.  This incident will be reported.

It seems I have to restart the system.

 Web browser

Why Firefox has been rebranded as iceweasel.

"Debian was initially given permission to use the trademarks, and adopted the Firefox name. However, because the artwork in Firefox had a proprietary copyright license which was not compatible with the Debian Free Software Guidelines, the substituted logo had to remain."

VIM commands

Help

• :help  -  vim help
• :help commandname - help on a particular command
• CTRL+] - jump to a highlighted topic
• CTRL+T - jump backwards

Motion

• :help left-right-motion
• j,k move up down
• h,l move left right
• b,w move previous or next word
• ctrl+b, ctrl+d move page up or page down

Undo redo

• u: undo last change (can be repeated to undo preceding commands)
• Ctrl-R: Redo changes which were undone (undo the undos). 
• Compare to '.' to repeat a previous change, at the current cursor position. Ctrl-R will redo a previously undone change, wherever the change occurred. 

Switch between navigation and editing mode

• A - move to the end of the line and switch to editing mode 
• I - switch to editing mode at the current place
• Escape - switch to navigation mode
• alt+h alt+j alt+k alt+l - switch to navigation mode and move
• alt+: - switch to navigation mode and send a command

Search and replace characters

• "/" to search, then n forward and N backwards
• "*" to search for the word under the cursor (you can also create a mapping to search for a selected text)

Vim wiki on search and replace 

• :s/foo/bar/g Find each occurrence of 'foo' (in the current line only), and replace it with 'bar'. 

• :%s/foo/bar/g Find each occurrence of 'foo' (in all lines), and replace it with 'bar'.

• %s/option value=".*"//g remove all beginnings of line. :%s/\option\n/, /g replace all end of line by comma + space. This cleans an html list of species for inclusion in a text.

Markdown

Display a list of first level header in a markdown document (found in quick markdown navigation/TOC)

:g/^# /#

Then enter the line number to jump to that line.

Line numbers

 Display line numbers

:set nu

Disable line numbers

:set nonu

Editing a whole line

• dd to delete a whole line
• yy to copy a whole line
• p to paste the copied or deleted text after the current line or 
• P to paste the copied or deleted text before the current line 

Copy, cut and paste

• Position the cursor where you want to begin cutting.
• Press v (or upper case V if you want to cut whole lines).
• Move the cursor to the end of what you want to cut.
• Press d to cut or y to copy.
• Move to where you would like to paste.
• Press P to paste before the cursor, or p to paste after. 

Indentation

Indentation replaced by spaces, add this to the ~/.vimrc file

set tabstop=4
set expandtab
set softtabstop=4
set shiftwidth=4
filetype indent on 

More details on vim indentation in the python wiki.

Multiple files and windows

• :e filename - edit another file 
• :ls         - show current buffers
• :b 2        - open buffer #2 in this window
• :b filename - open buffer #filename in this window
• :bd         - close the current buffer (! to forget changes)
• :bd filename -close a buffer by name 

Windows

• :sp[lit] filename  - split window and load another file
• :vs[plit] - same but split vertically  
• ctrl-w up arrow - move cursor up a window
• ctrl-w ctrl-w   - move cursor to another window (cycle)
• ctrl-w_         - maximize current window
• ctrl-w=         - make all equal size
• CTRL+z - suspend the process and get back to the shell
• fg - get back to vim

Vimdiff

View differences between file1 and file2 (vim documentation)

vimdiff file1 file2

spell check

Set spell check only in the local buffer:

:setlocal spell spelllang=en_gb  

 Turn spell check off

:set nospell

Mark word as correct, this creates a spell file under /home/user/.vim/spell:

zg

Mark word as incorrect

zw

Plugins for programming languages

• Vim R
• Vim on Python wiki (set softtabstop=4) or your  python development environment (set tabstop=4 and set softtabstop=4)

.vimrc

Text colour.
Add syntax highlight to your .vimrc

syntax enable

How to add a file extension to vim syntax highlight

au BufNewFile,BufRead *.dump set filetype=sql

I used it to display markdown files as text files:

au BufNewFile,BufRead *.md set filetype=txt

Debian GNU-Linux installation on a Lenovo T400

I needed an operating system more stable than Microsoft windows for my daily tasks such as: programming with the R statistical software and writing PDF reports with the Lyx document processor. I read about John MacFarlane (author of pandoc) who is using Debian with the xmonad desktop. This blogger documented why he switched from Ubuntu to Debian. The same blogger also wrote interesting posts at the electronic Frontier Fondation on privacy issues with Ubuntu: on the one side Ubuntu offers an easy way to install full Hard Drive encryption, on the other side, Ubuntu's default desktop search sends search requests over unencripted internet (I should move this additional content to another post).

15 years ago already, I had talked with a friend who was using Debian. I wonder if he still is?

I decided to give Debian a try.

Creating a bootable USB stick

• On a windows machine, I have downloaded the latest Debian live iso (I chose the .iso with the Gnome desktop). I  created a bootable USB stick with the help of Linux Live USB creator. 
• Alternatively, on a Debian machine, for another installation later I choose the Debian Net install version. Then according to Debian Instructions on how to prepare a USB stick, I copied it to the USB with the command :
• cp debian-7.6.0-amd64-netinst.iso /dev/sdb1
• sync
• Unfortunately this net install didn't work for me. Luckily, the live USB install worked fine.

Trying the live version from a USB stick

I introduced the USB key in the laptop,  pressed the "blue thinkvantage" key, entered the BIOS setup and changed the boot order. I placed the USB devices first in the list. The live version seemed to work fine, the Gnome desktop was responsive enough on that machine (Lenovo Thinkpad T400). The external screen was easy to set-up, and network access configured automatically. So I decided to install that Debian system on the hard drive.

Installation

I restarted the laptop with the USB key inserted. I chose the graphical installer. There was a small issue with the fact that the installer was looking for a CD-ROM. But no CD-ROM was available because I was running the install from a USB stick. Based on this blog post, I jumped to the command line interface and typed:

mount /dev/sdb /cdrom 

This command fixed the issue, installation could continue. I partitionned my hard drive to leave 100 GB for the windows partition and the rest (144 GB) for the new GNU-Linux system. Partitioning took a frightening amount of time. During which I thought the system as frozen. But after maybe an hour, installation carried on... I restarted the system. I am now writing this blog post from a fresh Debian system.

Hardware support

Docking station

Hotpluging usb keyboard was working on the laptop itself, but not working on the docking station. A quick fix was to disconnect and reconnect the docking station.
I have connected the docking station to an external larger screen I set this up as my main monitor under system settings / displays.

Different Keyboard layouts

My laptop has a Finnish keyboard, and I am located in France, therefore I use a French keyboard. Under system settings / region and language settings I could add the French keyboard layout. The fact that it's possible to "use the same layout for all windows" is a plus compared to windows. I regularly switch from the Finnish to the French keyboard layout. In windows, I had to change the keyboard layout for all windows one by one.

Network drive

From the Nautilus file manager, I could access windows shared drive using my company's internal network login details.
I have been trying to synchronize this folder using Unison (GUI for rsync). Setting up a local synchronization requires to mount the windows share in the file system. mount.cifs from cifs utils can mount windows shared drives.

Network printer

Under system settings /  printers, the network printer appears. But I didn't manage to connect to the Sharp MX 2600 N.

• MX 2600 N is not in the list of PDD files for sharp printers
• I found a PDD file for MX 2600N on a TU München site.

Wireless card

Show all hardware

lspci

 Network controller: Intel Corporation PRO/Wireless 5100 AGN

Install support for wifi device on Debian

apt-get install firmware-iwlwifi

Reinsert this module to access installed firmware:

# modprobe -r iwlwifi ; modprobe iwlwifi

Wifi works! :-)

Location of software packages

A mirror for France is located at http://ftp.fr.debian.org/debian/.

Install research software 

See also other software install tips in this post on Debian GNU Linux
Lyx editor for Latex

sudo aptitude install lyx

Jabref

sudo aptitude install jabref

Evolution with Microsoft Exchange plugin

My Company uses microsoft exchange on its email servers. An "exchange" plugin can be installed from the Synaptic package manager but this only works for exchange 200 and 2003. This site recommends using the mapi plugin instead which should work for exchange 2007 and 2010. As explained here, my company also uses a different server for the web mail access and the internal connection to the mail server.

To find the internal sever name, I went to the windows version of outlook 2007. Right click on mail box / properties / advanced ... / Microsoft Exchange server.

Mail loads fine now. I can send email too.

Calendar events are not displayed, even though outlook calendar events appear within the desktop calendar (The little calendar that appears when clicking on the desktop clock). This page says that I should upgrade to a more recent version of Evolution. This would require the use of Debian backports. Couldn't get backports to work for now. It said that I already have the most recent version of evolution installed.
I add this line to the 

# Backports repository
deb http://ftp.debian.org/debian wheezy-backports main contrib non-free

Ran this command

sudo apt-get update
sudo apt-get -t wheezy-backports install evolution

Got this message

evolution is already the newest version.

Well it's actually normal because evolution is currently (August 2014) not in the list of Debian backports.

R and Rstudio install on Debian wheezy 7

Instruction below kept here for historical purposes, I'll update these R specific instruction in another blog post.

I used the Synaptic package manager to add the R repository for Debian from a nearby mirror, under : settings / repositories / other software / add.
Add this APT line:

deb http://cran.univ-paris1.fr/bin/linux/debian/ wheezy-cran3/

There was an error:

W: GPG error: http://cran.univ-paris1.fr wheezy-cran3/ Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 06F90DE5381BA480

After looking at several forums, and this stackoverflow question, I installed debian-keyring and added the key with the commands:

gpg --keyserver pgpkeys.mit.edu --recv-key 06F90DE5381BA480
gpg -a --export 06F90DE5381BA480 |sudo apt-key add -

I could then install R version 3 from the synaptic package manager.

I downloaded R-studio and installed it. There was a missing dependency for libjpeg62. I installed that package from Synaptic. Then ran the dpkg command to install rstudio.

dpkg -i rstudio-0.98.507-i386.deb

Then I installed Git in order to clone my R project from an online repository.

git clone  project_repository_url

Within Rstudio, I installed a few packages:

install.packages(c("plyr", "reshape2", "ggplot2"))
install.packages(c("xtable", "markdown", "devtools"))

Ready to work!

Further reading

Wiki on installing Debian on Thinkpad laptops.

Git config on GNU-Linux

Colors

To add coloration:

git config --global color.ui true

SSH

Connect to a remote repository without using the password (ssh key). On Bitbucket, different connection protocols have different repository URL formats. Create a private and public ssh key pair with for example the seahorse key manager. Then add the public key to bitbucket under manage account / security / SSH keys.

Moving from https to ssh url format 

My previous connection was through https. You can skip this command if you start with a fresh repository.  Rename the remote "origin" to "originhttps".

git remote rename origin originhttps

Just wanted to be able to connect through https in case the ssh connection doesn't work. This remote tracking branch can be deleted later with:

git remote remove originhttps

Tell git to use a ssh:// remote

Then tell git to use a remote (the "ssh://" protocol name is optional as shown in this how-to):

git add remote origin ssh://git@bitbucket.org/accountname/reponame.git

Fetch the new remote tracking branch into the local repository. (It will probably not load anything new but is required to set this branch as upstream later):

git fetch origin master

Set upstream branch

The commands "Git pull" and "git push" still connected to the https url (which prompts for a password). I had to set the new remote as the default for my master branch:

git branch --set-upstream-to=origin/master

List remote branches (for information)

git branch -r

The repository is now ready to fetch, merge and pull on remote origin without entering the password.


See also my post on git commands.