Patterns

Sometimes it helps to think in terms of design patterns.

10 years ago, a friend of mine offered me a book on architectural patterns by Christopher Alexander (A Pattern Language: Towns, Buildings, Construction) I remember beautifully simple description of architectural patterns in buildings such as: a place by the window (inside a house) or a "high place" to look around town, or avoid X junctions, keep only T junctions in residential areas.

How about statistical patterns?

Including R data and plots in a Latex document with knitr

knitr default typesetting:

"The chunk option out.width is set to '\\maxwidth' by default if the output format is LaTeX."

Read part of an Excel sheet into an R data.frame

Documentation of the read.xlsx function: 

read.xlsx(file, sheetIndex, sheetName=NULL, rowIndex=NULL,  startRow=NULL, endRow=NULL, colIndex=NULL,  as.data.frame=TRUE, header=TRUE, colClasses=NA,  keepFormulas=FALSE, encoding="unknown", ...)

Returns a data.frame.
Example of use:

dtf <- file="filename,<!-----" read.xlsx="">
    sheetName = sheetname,
    rowIndex = 2:10, 
    colIndex = 5:20)

Building an R package

A package can be seen as a coherent group of functions available for future projects. Building your own package enables you to reuse and share your statistical procedures. Function parameters and examples can be documented with Roxygen to facilitate digging back into the code later on. I created my second package  based on instructions from Hadley. My package structure is composed of the following folders:

• R/   contains R code
• test/  contains tests
• inst/  contains files that will be exported with the package
• docs/  contains .Rmd documents illustrating code development steps and data analysis.
• data/    contains data sets exported with the package
• data-raw/    contains raw dataset and R code to extract raw data from disk and from web resources.

Code

Create a directory containing a package skeleton

devtools::create("packagename")

RStudio has a menu build / configure build tools where devtools package functions and document generation can be linked to keyboard shortcuts:

• document CTRL + SHIFT + D 
• build and reload CTRL + SHIFT + B

devtools::load_all() or Cmd + Shift + L, reloads all code in the package.
Add packages to the list of required packages devtools::use_package("dplyr") devtools::use_package("ggplot2", "suggests")

Data

For data I followed his recommendations in r-pkgs/data.rmd devtools::use_data(mtcars) devtools::use_data_raw() # Creates a data-raw/ folder and add it to .Rbuildignore

Tests

Example of testing for the devtools package

Bash command to build and check a package

Bash command to build a package directory:

R CMD build packagename

Bash command to check a package tarball:

R CMD check packagename_version.tar.gz

 An error log (good luck for understanding it) is visible at:

packagename.Rcheck/00check.log

Generate the documentation and check for documentation specific errors

R CMD Rd2pdf tradeflows --no-clean

The --no-clean option keeps among other files a temporary Latex which can be inspected under:

packagename.Rcheck/packagename-manual.tex

Alternatively the build and check procedure can be run in RStudio as explained above.

Debian GNU-Linux installation on a HP EliteBook

A nice laptop provided by my employer, although the name "EliteBook" sounds like marketing rubbish. My colleague installed Debian 8 on the system, as a dual boot with windows 8.
See also my earlier posts tagged Debian.

Multi boot

The system is a dual boot with Windows. Debian was installed with a traditional - non UEFI - boot loader. Therfore  I have to press F9 each time I want to start Debian. This remains a minor annoyance since I usually restart only every 2 weeks or so (often when the battery has fully drained after a trip). On ordinary days, I put the system in sleep mode in the evening and it awakens in Debian again in the morning.

HP boot instruction didn't work to change the boot order.

 Language

The Debian language was set to French, change back to English.
Reconfigure locales:
As root, run :

dpkg-reconfigure locale

User privileges

as a super user

apt-get install sudo

Add a user to the super user group

adduser username sudo

Log out and log back in for this change to take effect. Now the user should be in the sudo group.

Wireless card

To use the intel non free wireless drivers iwlwifi, in the synapticpackage manager settings / repositories, add "main contrib non-free" to the sections of Jessie packages. Then install firmware-iwlwifi.
In a terminal, restart the modules

 modprobe -r iwlwifi ; modprobe iwlwifi 

Wireless network Eduroam

security: WPA & WPA2 entreprise
Authentication: Tunneled TLS
CA certificate: /etc/ssl/certs/addtrust_external_root.pem
Inner authentication: PAP
Username: user_name@institution.fr
Password: ******

Email and calendar with Evolution

I configured email. I use 2 step authentication and had to generate an application password to load events from my private Google calendar. To add a public Google calendar, I added an "on the web calendar" and inserted the ical link for that public calendar.

Under Edit / Preferences / Contacts / Automatic contacts / I selected to create an automatic contact when sending an email.

R and R studio

Since a previous post on R in Debian, I have changed Debian version from Wheezy to Jessie. Luckily, Jessie contains a recent version of R,I don't need to add the Cran repositories anymore.

Installed Rstudio.

Installed the folowing packages (at a R prompt or in RStudio):

install.packages(c("plyr", "reshape2", "ggplot2"))
install.packages(c("xtable", "markdown", "devtools"))

Install RmySQL, requires mysql development library. At the Shell:

sudo apt-get install libmysqlclient-dev

In R :

install.packages("RMySQL")

Installing R package xlsx requires the java development kit. In the system shell:

sudo apt-get install openjdk-7-jdk

sudo R CMD javareconf

Then in R:

install.packages("xlsx")

Firefox rebranded as iceweasel  + Adblock plus

Installed adblock plus.

Installed icedove to access outlook email backup

See my previous post on accessing outlook email archive from Debian.

Debian dist-upgrade from Wheezy to Jessie

I needed a couple of recent software versions (Lyx version> 2.1 and Latex), and they were not available in the wheezy-backports, therefore I decided to upgrade from Debian Wheezy, the current stable version to Debian Jessie, the testing version. Debian Install FAQ.

This page gives the configuration changes and commands to be used:

1. Edit /etc/apt/sources.list and replace all occurrences of "wheezy" with "jessie".
2. run apt-get update
3. run apt-get upgrade
4. run apt-get dist-upgrade

For several reasons the upgrade process didn't complete properly. After a system reboot, there was no networking, no graphical desktop and even after starting the Gnome desktop by hand with "startx", there were several issues with system settings, such as printer setup.

Networking

I had an issue with networking not working.
According to the debian page on network setup, changes below are not recommended (see below).

I fixed by editing/etc/network/interfaces

# Lines added from odoepner.wordpress.com
auto eth0 

allow-hotplug eth0

iface eth0 inet dhcp

Then ran as root

etc/init.d/networking start

And network was working again.

This page explains how to put this network as a managed interface again: 

But I shouldn't have done this.
According to the debian page on network setup: "Keep configuration of "/etc/network/interfaces" as simple as in the following".
auto lo
iface lo inet loopback

No desktop manager at startup

I realised that there was no desktop manager at startup. I first followed answers to this question and edited  /etc/inittab. But it is not needed, because the boot manager systemd doesn't look into that configuration file.
I spent some time reading about a systemd controversy in Debian and why it doesn't matter so much in the end.

For the moment I start the Gnome desktop from a terminal with the command:

xstart

It looks like not all package had been upgraded.
I ran as root

apt-get upgrade -f

The following packages will be upgraded:
  live-tools

[...]

Preparing to unpack .../live-tools_4.0.2-1_all.deb ...
dpkg-divert: error: rename involves overwriting `/usr/bin/uptime' with
  different file `/usr/bin/uptime.orig.procps', not allowed

From Synaptic, I did a complete removal of the package live-tools.
After that many configuration steps took place. Printer setup utility was working again. And now there only remains a bluetooth issue.

Bluetooth issue

dpkg: error processing package gnome-bluetooth (--configure):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 bluez
 bluetooth
 gnome-bluetooth

Lyx upgraded to version 2.1.2

At least Lyx was upgraded to version 2.1.2. I had to "reconfigure LyX with Tools→Reconfigure;".Then I could create documents with Lyx again. And the R integration worked as well.

Upgrade might not have been needed

Afterwards, I realised that an upgrade might not have been needed:

"If you want to install a single package in Debian, you do not need to update the whole system. It can be done with three commands by inserting the repos for Testing and Unstable in /etc/apt/sources.list, fixing the distro you (mostly) wish by setting APT::Default-Release "stable" in /etc/apt/apt.conf.d/local, then doing aptitude install packagae_name/testing -t testing"