Commit 775dfe80 authored by Arnaud Legrand's avatar Arnaud Legrand

Initial import from github resources

parent de8e1799
# -*- mode: org -*-
#+TITLE: Emacs/org-mode
#+AUTHOR: Arnaud Legrand
#+DATE: June, 2018
#+STARTUP: overview indent
* Table of Contents :TOC:
- [[#installing-emacs-org-mode-ess-and-auctex][Installing emacs, org-mode, ess, and auctex.]]
- [[#os-specific-installation-procedure][OS-specific installation procedure]]
- [[#a-simple-reproducible-research-emacs-configuration-][A simple "/reproducible research/" emacs configuration ]]
- [[#emacs-tips-and-tricks][Emacs Tips and tricks]]
- [[#cheat-sheets][Cheat-sheets]]
- [[#video-tutorials][Video tutorials]]
- [[#additional-useful-emacs-packages][Additional useful emacs packages]]
- [[#other-resources][Other resources]]
- [[#a-replicable-article][A replicable article]]
- [[#a-stub-of-replicable-article][A stub of replicable article]]
* Installing emacs, org-mode, ess, and auctex.
** OS-specific installation procedure
*** Linux (Debian, Ubuntu)
We provide here only instructions for debian-based distributions. Feel
free to contribute to this document to provide up-to-date information
for other distributions (e.g.n redhat, fedora).
Today, the stable versions of the most common distributions provide
recent enough versions of emacs and org-mode:
- Debian (stretch) ships with [[https://packages.debian.org/stretch/emacs25][emacs 25.1]] and [[https://packages.debian.org/stretch/org-mode][org-mode 9.0.3]]
- Ubuntu (bionic 18.04) ships with [[https://packages.ubuntu.com/bionic/emacs25][emacs 25.2]] and [[https://packages.ubuntu.com/bionic/org-mode][org-mode 9.1.6]]
- Ubuntu (artful 17.04) ships with [[https://packages.ubuntu.com/artful/emacs25][emacs 25.2]] and [[https://packages.ubuntu.com/artful/org-mode][org-mode 9.0.9]]
If your distribution is older than this, well, it may be a good time
for upgrading...
Simply run (as root):
#+begin_src sh :results output :exports both
apt-get update ; apt-get install emacs25 org-mode ess r-base auctex
#+end_src
Then make sure you have a sufficiently recent version of emacs.
#+begin_src sh :results output :exports both
emacs --version 2>&1 | head -n 1
#+end_src
#+RESULTS:
: GNU Emacs 25.2.2
Likewise, you'll want to check you have a recent version of org-mode:
#+begin_src sh :results output :exports both
emacs -batch --funcall "org-version" 2>&1 | grep version
#+end_src
#+RESULTS:
: Org mode version 9.1.11 (9.1.11-dist @ /usr/share/emacs/25.2/site-lisp/elpa/org-9.1.11/)
The version numbers you get will depend on the distribution you are
running. _You really want to make sure you do not rely on org-mode 8_,
which is now deprecated.
*** Mac OS X
- *Option 1*: Install the =.dmg= file from [[http://vgoulet.act.ulaval.ca/][Vincent Goulet]]:
https://vigou3.github.io/emacs-modified-macos/. It ships with recent
versions:
- Emacs 26.1
- Org-mode 9.1.13
- ESS 17.11
In that case you may have several versions of emacs installed on
your machine and launching emacs from a terminal may not run the one
you just installed. If anyone know how to get rid of this...
- *Option 2*: If you already use =brew=, simply do the following
#+begin_src sh :results output :exports both
brew update
brew install emacs [--with-cocoa]
brew linkapps emacs
brew tap homebrew/science
brew install ess
brew install auctex # The name may be slightly more complicated here
brew install wget
#+end_src
*** Windows
Install the =.exe= file from [[http://vgoulet.act.ulaval.ca/][Vincent Goulet]]:
https://vigou3.github.io/emacs-modified-windows/. It ships with recent
versions:
- Emacs 26.1
- Org-mode 9.1.13
- ESS 17.11
** A simple "/reproducible research/" emacs configuration
Emacs comes with very basic default configuration and it appears like
everyone has its own taste. You will for example find [[https://www.emacswiki.org/emacs/StarterKits][here]] several
default Emacs configurations that reflect the preferences of their
creators. Likewise the configuration of Org-Mode is incredibly
flexible (see for example [[https://orgmode.org/worg/org-configs/index.html][the org-mode website]] for more
references). In the context of this MOOC, we propose you a relatively
minimalistic one that is rather "/reproducible research/" oriented by
adding a few org-mode specific configurations.
*** Step 0: Backup and download our configuration
The procedure we propose will wipe your already existing custom emacs
configuration if you already have one. *You should thus beforehand make
a backup of =~/.emacs= and of =~/.emacs.d/init.el=* (if these file
exists).
#+begin_src shell :results output :exports none
export FILE_LIST="rr_org/init.el rr_org/journal.org"
tar zcf rr_org_archive.tgz $FILE_LIST
#+end_src
#+RESULTS:
Then download [[file:rr_org_archive.tgz][this archive]] and uncompress it. It contains the
following files and we will refer to them in the following:
#+begin_src shell :results output :exports results
tar tzf rr_org_archive.tgz
#+end_src
#+RESULTS:
: rr_org/init.el
: rr_org/journal.org
*** Step 1: Prepare your journal
Create an =org/= directory in the top of your home:
#+begin_src sh :results output :exports both
mkdir -p ~/org/
#+end_src
Then copy =rr_org/journal.org= file in your =~/org/= directory. This
file will be your laboratory notebook and all the notes you will
capture with =C-c c= will go automatically go in this file. The first
entry of this notebook is populated with [[file:rr_org/journal.org::*Org-mode][many emacs shortcuts]] that you
should give a try.
*** Step 2: Set up Emacs configuration
Copy =rr_org/init.el= in your =~/.emacs.d/= directory.
Alternatively, if you do not want to mess with your already existing
emacs configuration, you may launch emacs with this specific
configuration with the following command: =emacs -q -l rr_org/init.el=.
*** Step 3: Check whether the installation is working or not
Open a new instance of Emacs and open a =foo.org= file. Copy the
following lines in this file:
: #+begin_src shell :session foo :results output :exports both
: ls -la
: #+end_src
Put your cursor inside this code block and execute it with the
following command: =C-c C-c= (If you are not familiar with Emacs
commands, this one means '=Ctrl + C=' twice)
A =#+RESULTS:= block with the result of the command should appear if it
worked.
In the video, we already have demonstrated the main features and
shortcuts of emacs/org-mode that will help you maintain a document and
benefit from literate programming. The list of features and shortcuts
is demonstrated in the [[file:rr_org/journal.org::*Emacs%20shortcuts][first entry of your labbook]].
*** Step 4: Open and play with your journal:
In step 1, you were told to create an journal in
=~org/journal.org=. First you probably want to make sure this file is
backed up in a revision control system like git. We leave it up to set
this up but if you have any trouble, feel free to ask on the FUN
forums.
* Emacs Tips and tricks
** Cheat-sheets
Learning emacs and org-mode can be difficult as there is an inordinate
amount of shortcuts. Many people have thus come up with
cheat-sheats. Here is a selection in case it helps:
*** Emacs
- [[file:org/journal.org::*Emacs%20shortcuts][Common and step-by-step emacs shortcuts for our /reproducible research/ configuration]]
- [[https://www.gnu.org/software/emacs/refcards/pdf/refcard.pdf][The official GNU emacs refcard]]
- Two graphical cheat-sheats by Sacha Hua on [[http://sachachua.com/blog/wp-content/uploads/2013/05/How-to-Learn-Emacs-v2-Large.png][how to learn emacs]] and on
[[http://sachachua.com/blog/wp-content/uploads/2013/08/20130830-Emacs-Newbie-How-to-Learn-Emacs-Keyboard-Shortcuts.png][how to learn emacs shortcuts]].
*** Org-mode
- [[file:org/journal.org::*Org-mode][Common and step-by-step org-mode shortcuts for our /reproducible research/ configuration]]
- [[https://orgmode.org/worg/orgcard.html][The official org-mode refcard]]
- [[https://orgmode.org/worg/dev/org-syntax.html][The official description of the org-mode syntax]] and a [[https://gist.github.com/hoeltgman/3825415][relatively concise description of the org-mode syntax]].
** Video tutorials
For those of you who prefer video explanations, here is a [[https://www.youtube.com/playlist?list=PL9KxKa8NpFxIcNQa9js7dQQIHc81b0-Xg][Youtube
channel with many step by step emacs tutorials]].
** Additional useful emacs packages
*** Company-mode
[[http://company-mode.github.io/][Company-mode]] is a text completion framework for Emacs. It allows to
have smart completion in emacs for the most common languages. If you
feel this is needed, you should follow the instructions from the
official webpage: http://company-mode.github.io/
*** Magit
[[https://magit.vc/][Magit]] is an emacs interface for Git. It is very powerfull and we use
it on a daily basis but you should definitely understand what git does
behind the scene beforehand. If you feel this would be useful for you,
you should follow the instructions from here:
https://www.emacswiki.org/emacs/Magit
** Other resources
- [[https://orgmode.org/orgguide.pdf][The compact Org-mode Guide]]
- [[https://github.com/dfeich/org-babel-examples][Many examples illustrating the use of different languages in org-mode]]
* A replicable article
** A stub of replicable article
Remember, you need a working LaTeX and R environment. If you can't
open a terminal and run the commands =R= and =pdflatex=, you will not be
able to generate this document. The article downloads the
corresponding LaTeX packages so you also need to have a working =wget=
command (alternatively, you can use =curl=). You may still read the
source ([[file:replicable_article/article.org][article.org]]) and understand how it works though.
#+begin_src shell :results output :exports none
make -C replicable_article/ replicable_article.tgz
#+end_src
#+RESULTS:
: make : on entre dans le répertoire « /home/alegrand/Work/Documents/Enseignements/RR_MOOC/references/module2/replicable_article »
: tar --xform "s|^|replicable_article/|" -zcf replicable_article.tgz article.org biblio.bib Makefile
: make : on quitte le répertoire « /home/alegrand/Work/Documents/Enseignements/RR_MOOC/references/module2/replicable_article »
Download the [[file:replicable_article/replicable_article.tgz][archive]], untar it and simply make to generate the
article. You should then be able to open the [[file:replicable_article/article.pdf][resulting article]]. This
is summarized in the following command:
#+begin_src sh :results output :exports both
wget --no-check-certificate -O replicable.tgz https://github.com/alegrand/RR_webinars/raw/master/1_replicable_article_laboratory_notebook/replicable/replicable.tgz ### Fix the broken URL
tar zxf replicable.tgz; cd replicable; make ; evince article.pdf
#+end_src
If the =make= command fails (especially on Mac), it may be because emacs
or something else is not correctly installed. In that case, open the
article directly with the following command:
#+begin_src sh :results output :exports both
emacs -q --eval "(setq enable-local-eval t)" --eval "(setq enable-local-variables t)" article.org
#+end_src
and export it to pdf with the following shortcut: =C-c C-e l o=
If it still doesn't work and emacs complains about not finding ESS, it
may be because you installed ESS in your home instead of
system-wide. In that case, try to remove the =-q= in the previous
command line to load your personal emacs configuration.
Finally, when you'll be tired of always reexecuting all the source
code when exporting, just look for the following line in [[file:replicable_article/article.org][article.org]]:
#+BEGIN_EXAMPLE
# #+PROPERTY: header-args :eval never-export
#+END_EXAMPLE
If you remove the =# = in the beginning of the line, it will not be a
comment anymore and will indicate org-mode to stop evaluating every
chunk of code when exporting.
# -*- mode: org -*-
#+TITLE: Rstudio
#+AUTHOR: Arnaud Legrand
#+DATE: June, 2018
#+STARTUP: overview indent
* Table of Contents :TOC:
- [[#interacting-with-gitlab][Interacting with gitlab]]
- [[#additional-jupyter-resources-or-tricks][Additional Jupyter resources or tricks]]
- [[#tips-and-tricks][Tips and tricks]]
- [[#running-r-and-python-in-the-same-notebook][Running R and Python in the same notebook]]
- [[#exporting-a-notebook][Exporting a notebook]]
- [[#installing-jupyter-or-jupyter-hub-on-your-own-machine][Installing jupyter or jupyter hub on your own machine]]
- [[#installing-jupyter][Installing jupyter]]
- [[#using-r][Using R]]
- [[#exporting-your-notebooks-with-latex][Exporting your notebooks with latex]]
- [[#interacting-with-gitlab][Interacting with gitlab]]
- [[#interesting-extensions][Interesting extensions]]
- [[#installing-jupyterhub][Installing jupyterhub]]
* Interacting with gitlab
#+BEGIN_QUOTE
This section needs some love.
#+END_QUOTE
As you may have seen from the videos, we provide two buttons to sync
with gitlab. Indeed, even if you recover your files when you log back
into fun, the jupyter runs on a different machine than the gitlab
server and you need to commit your notebook every time you have made a
significant step. This is also the only way to really make it public
and to share your notebook with others.
* Additional Jupyter resources or tricks
** Tips and tricks
The following [[https://www.dataquest.io/blog/jupyter-notebook-tips-tricks-shortcuts/][webpage]] lists several Jupyter tricks (in particular, it
illustrates many =Ipython magic= commands) that should improve your
efficiency (note that this blog post is about two years old so some of
the tricks may have been integrated in the default behavior of jupyter
now).
** Running R and Python in the same notebook
The best solution to this is to install rpy2. On my machine, I have
installed the =python3-rpy2= debian package with =apt-get install=. An
other alternative consists in going through the python package manager
with
#+begin_src python :results output :exports both
pip install rpy2
#+end_src
Then you'll be able to use both languages in the same notebook by:
1. Loading =rpy2=:
#+begin_src python :results output :exports both
%load_ext rpy2.ipython
#+end_src
2. Using the =%R= Ipython magic:
#+begin_src python :results output :exports both
%R
summary(cars)
#+end_src
Python objects can then even be passed to R as follows (assuming df
is a pandas dataframe:
#+begin_src python :results output :exports both
%%R -i df
plot((df)
#+end_src
** Exporting a notebook
Obviously, you can convert to html or pdf using the using the =File >
Download as > HTML= (or =PDF=) menu option. This can also be done from
the command line with the following command:
#+begin_src sh :results output :exports both
ipython3 nbconvert --to pdf Untitled.ipynb
#+end_src
If you want to use a specific style, then the nbconvert exporter
should be customised. This is discussed and demoed [[http://markus-beuckelmann.de/blog/customizing-nbconvert-pdf.html][here]]. We encourage
you to simply read the [[https://nbconvert.readthedocs.io/en/latest/][doc of nbconvert]].
Instead of going directly through LaTeX and playing too much with the
nbconvert exporter, an other option consists in exporting to Markdown
and playing with [[https://pandoc.org/][pandoc]]. Both approaches work, it's rather a matter of
taste.
* Installing jupyter or jupyter hub on your own machine
** Installing jupyter
Here is what you should install:
#+begin_src shell :results output :exports both
sudo apt-get install jupyter-notebook python3-pip python3-matplotlib python3-numpy
#+end_src
On my machine, I got the version 5.4.1:
#+begin_src shell :results output :exports both
jupyter-notebook --version
#+end_src
#+RESULTS:
: 5.4.1
The ipython notebook can then be run with the following command:
#+begin_src shell :results output :exports both
jupyter-notebook
#+end_src
** Using R
If you also want to have the R kernel in jupyter, follow [[https://github.com/IRkernel/IRkernel#installation][these
instructions]].
Alternatively, you can install =rpy2= (see the [[*Running R and Python in the same notebook][Running R and Python in
the same notebook]] section).
#+begin_src shell :results output :exports both
sudo apt-get python3-rpy2
#+end_src
** Exporting your notebooks with latex
Here is what I had to install to make sure the notebook export via
latex works:
#+begin_src shell :results output :exports both
sudo apt-get install wkhtmltopdf
sudo apt-get install texlive-xetex
#+end_src
** Interacting with gitlab
Here is our [[https://github.com/brospars/nb-git][jupyter extension that allows to git push/pull from the
notebooks]]:
#+begin_src shell :results output :exports both
jupyter nbextension install https://raw.githubusercontent.com/brospars/nb-git/master/nb-git.js
jupyter nbextension enable nb-git
#+end_src
** Interesting extensions
- [[https://stackoverflow.com/questions/33159518/collapse-cell-in-jupyter-notebook][Code folding]]
#+begin_src shell :results output :exports both
pip3 install jupyter_contrib_nbextensions
# jupyter contrib nbextension install --user # not done yet
#+end_src
- [[https://github.com/kirbs-/hide_code][Hiding code]]
#+begin_src sh :results output :exports both
sudo pip3 install hide_code
sudo jupyter-nbextension install --py hide_code
jupyter-nbextension enable --py hide_code
jupyter-serverextension enable --py hide_code
#+end_src
** Installing jupyterhub
#+BEGIN_QUOTE
Benoit may give a few hints on this.
#+END_QUOTE
ARCHIVE=replicable_article
# ARCHIVE_FILES=$(shell git ls-tree -r master --name-only | grep -v -e 'article.pdf')
ARCHIVE_FILES=article.org biblio.bib Makefile
all: article.pdf
%.tex: %.org
emacs -q -batch --eval "(setq enable-local-eval t)" --eval "(setq enable-local-variables t)" $^ --funcall org-latex-export-to-latex
%.pdf: %.tex
pdflatex $^
bibtex `basename $^ .tex`
pdflatex $^
pdflatex $^
%.html: %.org
emacs -q -batch --eval "(setq enable-local-eval t)" --eval "(setq enable-local-variables t)" $^ --funcall org-html-export-to-html
$(ARCHIVE).tgz: $(ARCHIVE_FILES)
tar --xform "s|^|$(ARCHIVE)/|" -zcf $@ $^
clean:
rm -f article.aux article.bbl article.blg article.log article.out *~
distclean: clean
rm -f article.html article.tex article.pdf figure.pdf data.csv measurements.txt IEEEtran.bst IEEEtran.cls \#article.org\#
# -*- coding: utf-8 -*-
# -*- mode: org -*-
#+TITLE: Org Mode Template for an IEEE Conference Article
#+AUTHOR: John Doe, John Smith, and Jane Doe
#+STARTUP: overview indent inlineimages logdrawer
#+LANGUAGE: en
#+OPTIONS: num:nil toc:t \n:nil @:t ::t |:t ^:nil -:t f:t *:t <:t
#+OPTIONS: TeX:t LaTeX:t skip:nil d:nil todo:t pri:nil tags:not-in-toc
#+OPTIONS: author:nil email:nil creator:nil timestamp:t
#+TAGS: noexport(n) deprecated(d)
#+EXPORT_SELECT_TAGS: export
#+EXPORT_EXCLUDE_TAGS: noexport
# # Default org-mode HTML style
# #+HTML_HEAD: <link rel="stylesheet" title="Standard" href="http://orgmode.org/worg/style/worg.css" type="text/css" />
# # Shiny readthedocs HTML style
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="http://www.pirilampo.org/styles/readtheorg/css/htmlize.css"/>
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="http://www.pirilampo.org/styles/readtheorg/css/readtheorg.css"/>
#+HTML_HEAD: <script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.3/jquery.min.js"></script>
#+HTML_HEAD: <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></script>
#+HTML_HEAD: <script type="text/javascript" src="http://www.pirilampo.org/styles/lib/js/jquery.stickytableheaders.js"></script>
#+HTML_HEAD: <script type="text/javascript" src="http://www.pirilampo.org/styles/readtheorg/js/readtheorg.js"></script>
# ### By default, all code chunks are being run when exporting. To
# ### avoid this, simply remove the "# " of the next line.
# #+PROPERTY: header-args :eval never-export
#+LATEX_CLASS: IEEEtran
#+LATEX_HEADER: \usepackage[T1]{fontenc}
#+LATEX_HEADER: \usepackage[utf8]{inputenc}
#+LATEX_HEADER: \usepackage{graphicx}
#+LATEX_HEADER: \usepackage{xspace,ifthen}
#+LATEX_HEADER: \usepackage{amsmath,amssymb}
#+LATEX_HEADER: \usepackage[american]{babel}
#+LATEX_HEADER: \usepackage{url} \urlstyle{sf}
#+BEGIN_EXPORT latex
\let\oldcite=\cite
\renewcommand\cite[2][]{~\ifthenelse{\equal{#1}{}}{\oldcite{#2}}{\oldcite[#1]{#2}}\xspace}
\let\oldref=\ref
\def\ref#1{~\oldref{#1}\xspace}
\sloppy
#+END_EXPORT
#+BEGIN_EXPORT latex
\title{Org Mode Template for an IEEE Conference Article}
%\subtitle{}
\author{%
\IEEEauthorblockN{%
John Doe\IEEEauthorrefmark{1},
John Smith\IEEEauthorrefmark{2},
Jane Doe\IEEEauthorrefmark{1},
}\smallskip
% \\
\IEEEauthorblockA{
\IEEEauthorrefmark{1}University of Middle-of-Nowhere; \url{firstname.lastname@mon.edu}}
\IEEEauthorblockA{
\IEEEauthorrefmark{2}Institute of Whatever; \url{firstname.lastname@smith.org}}
}
\maketitle
#+END_EXPORT
#+BEGIN_abstract
Reproducible research has become increasingly important, just like
parallel architectures so we propose a novel experimental study of a
parallel implementation of the quicksort algorithm that builds on
reproducible research technology.
#+END_abstract
* Getting the LaTeX packages :noexport:
#+begin_src shell :results output :exports both
set -e
if which wget; then
wget --no-check-certificate -O IEEEtran.cls http://mirror.ibcp.fr/pub/CTAN/macros/latex/contrib/IEEEtran/IEEEtran.cls
wget --no-check-certificate -O IEEEtran.bst http://mirrors.ctan.org/macros/latex/contrib/IEEEtran/bibtex/IEEEtran.bst
else
if which curs; then
curl -o IEEEtran.cls http://mirror.ibcp.fr/pub/CTAN/macros/latex/contrib/IEEEtran/IEEEtran.cls
curl -o IEEEtran.bst http://mirrors.ctan.org/macros/latex/contrib/IEEEtran/bibtex/IEEEtran.bst
else
echo "Neither wget nor curl was found!!! Get a real OS!" 1>&2
exit 1
fi
fi
#+end_src
* Getting the data :noexport:
#+begin_src shell :results output raw :exports both
wget --no-check-certificate -O measurements.txt https://raw.githubusercontent.com/alegrand/M2R-ParallelQuicksort/master/data/sama_2014-10-13/measurements_03%3A47.txt
echo 'file:measurements.txt'
#+end_src
* Extracting traces from data files :noexport:
#+begin_src perl :results output raw :exports both :var input="measurements.txt" output="data.csv"
use strict;
open INPUT, $input;
open OUTPUT, "> $output";
my($line);
my($size);
my($seq,$par,$libc);
print OUTPUT "Size, Seq, Par, Libc\n" ;
while($line=<INPUT>) {
chomp $line;
if($line =~/^Size: ([\d\.]*)$/) {
$size = $1;
next;
}
if($line =~/^Sequential quicksort.*: ([\d\.]*) sec.$/) {
$seq=$1; next;
}
if($line =~/^Parallel quicksort.*: ([\d\.]*) sec.$/) {
$par=$1; next;
}
if($line =~/^Built-in quicksort.*: ([\d\.]*) sec.$/) {
$libc=$1;
print OUTPUT "$size, $seq, $par, $libc\n";
next;
}
}
print "file:$output"
#+end_src
* R functions :noexport:
Initialization and reading of the data.
#+begin_src R :results output :session *R* :exports both
df=read.csv("data.csv",strip.white=T,header=T)
#+end_src
#+RESULTS:
Let's compute some (crappy) statistics.
#+begin_src R :results output :session *R* :exports both
sdf = data.frame()
for(size in unique(df$Size)) {
seq = mean(df[df$Size == size,]$Seq)
par = mean(df[df$Size == size,]$Par)
libc = mean(df[df$Size == size,]$Libc)
sdf = rbind(sdf,data.frame(Size=size,Seq=seq,Par=par,Libc=libc))
}
#+end_src
#+RESULTS:
* Introduction
With the advent of parallel architecture, it is tempting to propose
parallel implementations of classical operations. In this wonderful
article, we report the performance gain of the well known quicksort
algorithm, whose parallelization seems natural.
* Context
Quicksort (sometimes called partition-exchange sort) is an efficient
sorting algorithm, serving as a systematic method for placing the
elements of an array in order. Developed by Tony Hoare in 1959 with
his work published in 1961\cite{qsort}, it is still a commonly used
algorithm for sorting. Quicksort is a divide and conquer
algorithm. Quicksort first divides a large array into two smaller
sub-arrays: the low elements and the high elements. Quicksort can
then recursively sort the sub-arrays. We propose to sort the
sub-arrays in parallel on two threads and to bound the recursion
level.
* Related Work
Well, this is such a brilliant and novel idea that none has worked
on this cutting-edge topic. Parallel architecture barely existed in
1959, when Hoare proposed the sequential version of this
algorithm\cite{qsort}. Some colleagues mentioned some obscure
parallelization of the partitioning phase but it is too complex to
implement and certainly too costly to provide any performance gain.
* Methodology
We used a Dell Latitude 6430u with 16Gb of RAM running a Debian with
Linux 3.14.15. The CPU is an Intel(R) Core(TM) i7-3687U CPU @
2.10GHz comprising two physical cores and hyperthreading. The
=performance= frequency governor was used. We used FCC 5.3.1 with the
following compilation flags: =-g -Wall -O2 -pthread -lrt=. Since we
care a lot about reproducibility of our research, all the sources
and detailed information regarding our experiments are provided on
[[https://github.com/alegrand/M2R-ParallelQuicksort/blob/master/journal.org][Github]]
#+LaTeX:\footnote{Or figshare, or zenodo, or whatever platform you prefer!}.
We used the wonderful Org-mode
format\cite{schulte11:_activ_docum_org_mode} to both document our
experiments and create a replicable article that can be obtained on
[[https://github.com/alegrand/RR_webinars/blob/master/1_replicable_article_laboratory_notebook/replicable/article.org][Github]] as well. We could have used the Ipython
approach\cite{PER-GRA:2007} but we dishonestly ruled it out because
we wanted to be in full control of our C experimental setup and did
not know how to properly use it to write a replicable article.
Finally, it is important to explain that each measurement was
repeated five times and the set of experiments was absolutely not
randomized, which may compromise the validity of our observations
and conclusions.
* Experimental results
#+begin_src R :results graphics :file "figure.pdf" :exports none :width 6 :height 4 :session *R*
# This code will not appear. It is just here to generate a nice figure to include later on.
linetype <- c(1:3)
plotchar <- c(1:3)
plot(sdf$Size,sdf$Seq,log="x", type="b", lty=linetype[1], pch=plotchar[1])
legend( x="topleft",
legend=c("Sequential execution","Parallel execution", "Default Libc execution"),
lwd=1, lty=linetype, pch=plotchar)
lines(sdf$Size,sdf$Par, type="b", lty=linetype[2], pch=plotchar[2])
lines(sdf$Size,sdf$Libc, type="b", lty=linetype[3], pch=plotchar[3])
#+end_src
#+RESULTS:
[[file:figure.pdf]]
As we can see in Figure\ref{fig.comparison}, to benefit from the parallel
version of our quicksort implementation, significantly large arrays
(over a million entry) are required. Of course, it is difficult to
conclude from so few measurements and a better experiment design
would be useful.
#+LaTeX: \vspace{-1cm}
#+CAPTION: Comparing performances of several implementation of the quicksort algorithm\label{fig.comparison}
#+ATTR_LaTeX: :width .8\linewidth
file:figure.pdf
#+LaTeX: \vspace{-.3cm}
* Conclusion
Exploiting parallel machine can be quite difficult and tedious. From
our experience, such architecture are useful only when processing
sufficiently large data sets. As a future work, we intend to
consolidate our study with more experiments.
#+Latex:\section*{Acknowledgments}
This work is partially supported by the FOO and BAR
projects. Experiments presented in this paper were carried out using
our own experimental testbed with support of our university.
#+LaTeX: \nocite{*}
#+LaTeX: \def\raggedright{}
#+LaTeX: \bibliographystyle{IEEEtran}
#+LaTeX: \bibliography{biblio}
* Emacs Setup :noexport:
This document has local variables in its postembule, which should
allow Org-mode to work seamlessly without any setup. If you're
uncomfortable using such variables, you can safely ignore them at
startup. Exporting may require that you copy them in your .emacs.
# Local Variables:
# eval: (add-to-list 'load-path ".")
# eval: (require 'org-install)
# eval: (org-babel-do-load-languages 'org-babel-load-languages '((shell . t) (R . t) (perl . t) (python . t) ))
# eval: (setq org-confirm-babel-evaluate nil)
# eval: (unless (boundp 'org-latex-classes) (setq org-latex-classes nil))
# eval: (add-to-list 'org-latex-classes '("IEEEtran"
# "\\documentclass[conference, 10pt, compsocconf]{IEEEtran}\n \[NO-DEFAULT-PACKAGES]\n \[EXTRA]\n \\usepackage{graphicx}\n \\usepackage{hyperref}" ("\\section{%s}" . "\\section*{%s}") ("\\subsection{%s}" . "\\subsection*{%s}") ("\\subsubsection{%s}" . "\\subsubsection*{%s}") ("\\paragraph{%s}" . "\\paragraph*{%s}") ("\\subparagraph{%s}" . "\\subparagraph*{%s}")))
# eval: (setq org-alphabetical-lists t)
# eval: (setq org-src-fontify-natively t)
# eval: (setq ess-ask-for-ess-directory nil)
# eval: (setq ispell-local-dictionary "american")
# eval: (eval (flyspell-mode t))
# End:
@article{schulte11:_activ_docum_org_mode,
author ={Eric Schulte and Dan Davison},
title ={Active Document with Org-Mode},
journal ={Computing in Science \& Engineering},
year =2011,
volume =13,
number =3,
pages ={66--73},
month ={May/June}
}
@Article{PER-GRA:2007,
Author = {P\'erez, Fernando and Granger, Brian E.},
Title = {{IP}ython: a System for Interactive Scientific Computing},
Journal = {Computing in Science and Engineering},
Volume = {9},
Number = {3},
Pages = {21--29},
month = may,
year = 2007,
url = "http://ipython.org",
ISSN = "1521-9615",
doi = {10.1109/MCSE.2007.53},
publisher = {IEEE Computer Society},
}
@article{qsort,
author = {Hoare, C. A. R.},
title = {Algorithm 64: Quicksort},
journal = {Commun. ACM},
issue_date = {July 1961},
volume = {4},
number = {7},
month = jul,
year = {1961},
issn = {0001-0782},
pages = {321--},
url = {http://doi.acm.org/10.1145/366622.366644},
doi = {10.1145/366622.366644},
acmid = {366644},
publisher = {ACM},
address = {New York, NY, USA},
}
Size: 100
Sequential quicksort took: 0.000010 sec.
Parallel quicksort took: 0.004024 sec.
Built-in quicksort took: 0.000013 sec.
Size: 100
Sequential quicksort took: 0.000010 sec.
Parallel quicksort took: 0.004448 sec.
Built-in quicksort took: 0.000014 sec.
Size: 100
Sequential quicksort took: 0.000009 sec.
Parallel quicksort took: 0.003384 sec.
Built-in quicksort took: 0.000013 sec.
Size: 100
Sequential quicksort took: 0.000010 sec.
Parallel quicksort took: 0.003738 sec.
Built-in quicksort took: 0.000012 sec.
Size: 100
Sequential quicksort took: 0.000010 sec.
Parallel quicksort took: 0.003133 sec.
Built-in quicksort took: 0.000011 sec.
Size: 1000
Sequential quicksort took: 0.000128 sec.
Parallel quicksort took: 0.020407 sec.
Built-in quicksort took: 0.000209 sec.
Size: 1000
Sequential quicksort took: 0.000126 sec.
Parallel quicksort took: 0.022003 sec.
Built-in quicksort took: 0.000201 sec.
Size: 1000
Sequential quicksort took: 0.000128 sec.
Parallel quicksort took: 0.016149 sec.
Built-in quicksort took: 0.000210 sec.
Size: 1000
Sequential quicksort took: 0.000128 sec.
Parallel quicksort took: 0.014594 sec.
Built-in quicksort took: 0.000209 sec.
Size: 1000
Sequential quicksort took: 0.000129 sec.
Parallel quicksort took: 0.014905 sec.
Built-in quicksort took: 0.000210 sec.
Size: 10000
Sequential quicksort took: 0.001774 sec.
Parallel quicksort took: 0.018943 sec.
Built-in quicksort took: 0.001720 sec.
Size: 10000
Sequential quicksort took: 0.001698 sec.
Parallel quicksort took: 0.016226 sec.
Built-in quicksort took: 0.001733 sec.
Size: 10000
Sequential quicksort took: 0.001652 sec.
Parallel quicksort took: 0.017348 sec.
Built-in quicksort took: 0.001702 sec.
Size: 10000
Sequential quicksort took: 0.001680 sec.
Parallel quicksort took: 0.017302 sec.
Built-in quicksort took: 0.001726 sec.
Size: 10000
Sequential quicksort took: 0.001675 sec.
Parallel quicksort took: 0.017386 sec.
Built-in quicksort took: 0.001716 sec.
Size: 100000
Sequential quicksort took: 0.020040 sec.
Parallel quicksort took: 0.050548 sec.
Built-in quicksort took: 0.020300 sec.
Size: 100000
Sequential quicksort took: 0.020004 sec.
Parallel quicksort took: 0.043119 sec.
Built-in quicksort took: 0.020504 sec.
Size: 100000
Sequential quicksort took: 0.019763 sec.
Parallel quicksort took: 0.050735 sec.
Built-in quicksort took: 0.020439 sec.
Size: 100000
Sequential quicksort took: 0.019913 sec.
Parallel quicksort took: 0.049806 sec.
Built-in quicksort took: 0.020541 sec.
Size: 100000
Sequential quicksort took: 0.019726 sec.
Parallel quicksort took: 0.044636 sec.
Built-in quicksort took: 0.020252 sec.
Size: 1000000
Sequential quicksort took: 0.230648 sec.
Parallel quicksort took: 0.162221 sec.
Built-in quicksort took: 0.242869 sec.
Size: 1000000
Sequential quicksort took: 0.235778 sec.
Parallel quicksort took: 0.162137 sec.
Built-in quicksort took: 0.241607 sec.
Size: 1000000
Sequential quicksort took: 0.238383 sec.
Parallel quicksort took: 0.163279 sec.
Built-in quicksort took: 0.242786 sec.
Size: 1000000
Sequential quicksort took: 0.232921 sec.
Parallel quicksort took: 0.170237 sec.
Built-in quicksort took: 0.241583 sec.
Size: 1000000
Sequential quicksort took: 0.230096 sec.
Parallel quicksort took: 0.153896 sec.
Built-in quicksort took: 0.242492 sec.
(require 'org-install)
(require 'org)
(setq inhibit-splash-screen t)
(setq frame-title-format
'("Emacs - " (buffer-file-name "%f"
(dired-directory dired-directory "%b"))))
(global-font-lock-mode t)
(custom-set-faces
'(flyspell-incorrect ((t (:inverse-video t)))))
(line-number-mode 1)
(column-number-mode 1)
(load-library "paren")
(show-paren-mode 1)
(transient-mark-mode t)
(require 'paren)
(defalias 'yes-or-no-p 'y-or-n-p)
(set-terminal-coding-system 'utf-8)
(set-keyboard-coding-system 'utf-8)
(prefer-coding-system 'utf-8)
(setq
ns-command-modifier 'meta ; Apple/Command key is Meta
ns-alternate-modifier nil ; Option is the Mac Option key
ns-use-mac-modifier-symbols nil ; display standard Emacs (and not standard Mac) modifier symbols
)
(cua-mode t)
(defun jump-mark ()
(interactive)
(set-mark-command (point)))
(defun beginning-of-defun-and-mark ()
(interactive)
(push-mark (point))
(beginning-of-defun))
(defun end-of-defun-and-mark ()
(interactive)
(push-mark (point))
(end-of-defun))
(global-set-key "\^c\^b" 'beginning-of-defun-and-mark)
(global-set-key "\^c\^e" 'end-of-defun-and-mark)
(global-set-key "\^c\^j" 'jump-mark)
(global-set-key [S-f6] 'jump-mark) ;; jump from mark to mark
(global-set-key "\M-g" 'goto-line)
(global-set-key (kbd "C-+") 'text-scale-increase)
(global-set-key (kbd "C--") 'text-scale-decrease)
;; C-x C-0 restores the default font size
;; Inspired from http://tex.stackexchange.com/questions/166681/changing-language-of-flyspell-emacs-with-a-shortcut
;; (defun spell (choice)
;; "Switch between language dictionaries."
;; (interactive "cChoose: (a) American | (f) Francais")
;; (cond ((eq choice ?1)
;; (setq flyspell-default-dictionary "american")
;; (setq ispell-dictionary "american")
;; (ispell-kill-ispell))
;; ((eq choice ?2)
;; (setq flyspell-default-dictionary "francais")
;; (setq ispell-dictionary "francais")
;; (ispell-kill-ispell))
;; (t (message "No changes have been made."))) )
(define-key global-map (kbd "C-c s a") (lambda () (interactive) (ispell-change-dictionary "american")))
(define-key global-map (kbd "C-c s f") (lambda () (interactive) (ispell-change-dictionary "francais")))
(define-key global-map (kbd "C-c s r") 'flyspell-region)
(define-key global-map (kbd "C-c s b") 'flyspell-buffer)
(define-key global-map (kbd "C-c s s") 'flyspell-mode)
(global-set-key [f5] '(lambda () (interactive) (revert-buffer nil t nil)))
(defun auto-fill-mode-on () (TeX-PDF-mode 1))
(add-hook 'tex-mode-hook 'TeX-PDF-mode-on)
(add-hook 'latex-mode-hook 'TeX-PDF-mode-on)
(setq TeX-PDF-mode t)
(defun auto-fill-mode-on () (auto-fill-mode 1))
(add-hook 'text-mode-hook 'auto-fill-mode-on)
(add-hook 'emacs-lisp-mode 'auto-fill-mode-on)
(add-hook 'tex-mode-hook 'auto-fill-mode-on)
(add-hook 'latex-mode-hook 'auto-fill-mode-on)
(setq org-directory "~/org/")
(setq org-hide-leading-stars t)
(setq org-alphabetical-lists t)
(setq org-src-fontify-natively t) ;; you want this to activate coloring in blocks
(setq org-src-tab-acts-natively t) ;; you want this to have completion in blocks
(setq org-hide-emphasis-markers t) ;; to hide the *,=, or / markers
(setq org-pretty-entities t) ;; to have \alpha, \to and others display as utf8 http://orgmode.org/manual/Special-symbols.html
(global-set-key "\C-cl" 'org-store-link)
(global-set-key "\C-cc" 'org-capture)
(global-set-key (kbd "C-c a") 'org-agenda)
(define-key global-map "\C-cl" 'org-store-link)
(define-key global-map (kbd "C-c a") 'org-agenda)
(global-set-key "\C-cb" 'org-iswitchb)
(setq org-default-notes-file "~/org/notes.org")
(define-key global-map "\C-cd" 'org-capture)
(setq org-capture-templates (quote (("t" "Todo" entry (file+headline "~/org/liste.org" "Tasks") "* TODO %?
%i
%a" :prepend t) ("j" "Journal" entry (file+datetree "~/org/journal.org") "* %?
Entered on %U
%i
%a"))))
(setq org-agenda-include-all-todo t)
(setq org-agenda-include-diary t)
(global-set-key (kbd "C-c d") 'insert-date)
(defun insert-date (prefix)
"Insert the current date. With prefix-argument, use ISO format. With
two prefix arguments, write out the day and month name."
(interactive "P")
(let ((format (cond
((not prefix) "** %Y-%m-%d")
((equal prefix '(4)) "[%Y-%m-%d]"))))
(insert (format-time-string format))))
(global-set-key (kbd "C-c t") 'insert-time-date)
(defun insert-time-date (prefix)
"Insert the current date. With prefix-argument, use ISO format. With
two prefix arguments, write out the day and month name."
(interactive "P")
(let ((format (cond
((not prefix) "[%H:%M:%S; %d.%m.%Y]")
((equal prefix '(4)) "[%H:%M:%S; %Y-%m-%d]"))))
(insert (format-time-string format))))
(global-set-key (kbd "C-c l") 'org-store-link)
(global-set-key (kbd "C-c <up>") 'outline-up-heading)
(global-set-key (kbd "C-c <left>") 'outline-previous-visible-heading)
(global-set-key (kbd "C-c <right>") 'outline-next-visible-heading)
;; In org-mode 9 you need to have #+PROPERTY: header-args :eval never-export
;; in the beginning or your document to tell org-mode not to evaluate every
;; code block every time you export.
(setq org-confirm-babel-evaluate nil) ;; Do not ask for confirmation all the time!!
(org-babel-do-load-languages
'org-babel-load-languages
'(
(shell . t)
(python . t)
(R . t)
(ruby . t)
(ocaml . t)
(ditaa . t)
(dot . t)
(octave . t)
(sqlite . t)
(perl . t)
(screen . t)
(plantuml . t)
(lilypond . t)
(org . t)
(makefile . t)
))
(setq org-src-preserve-indentation t)
(add-to-list 'org-structure-template-alist
'("s" "#+begin_src ?\n\n#+end_src" "<src lang=\"?\">\n\n</src>"))
(add-to-list 'org-structure-template-alist
'("m" "#+begin_src emacs-lisp :tangle init.el\n\n#+end_src" "<src lang=\"emacs-lisp\">\n\n</src>"))
(add-to-list 'org-structure-template-alist
'("r" "#+begin_src R :results output :session *R* :exports both\n\n#+end_src" "<src lang=\"R\">\n\n</src>"))
(add-to-list 'org-structure-template-alist
'("R" "#+begin_src R :results output graphics :file (org-babel-temp-file \"figure\" \".png\") :exports both :width 600 :height 400 :session *R* \n\n#+end_src" "<src lang=\"R\">\n\n</src>"))
(add-to-list 'org-structure-template-alist
'("RR" "#+begin_src R :results output graphics :file (org-babel-temp-file (concat (file-name-directory (or load-file-name buffer-file-name)) \"figure-\") \".png\") :exports both :width 600 :height 400 :session *R* \n\n#+end_src" "<src lang=\"R\">\n\n</src>"))
(add-to-list 'org-structure-template-alist
'("p" "#+begin_src python :results output :exports both\n\n#+end_src" "<src lang=\"python\">\n\n</src>"))
(add-to-list 'org-structure-template-alist
'("P" "#+begin_src python :results output :session *python* :exports both\n\n#+end_src" "<src lang=\"python\">\n\n</src>"))
(add-to-list 'org-structure-template-alist
'("PP" "#+begin_src python :results file :var matplot_lib_filename=(org-babel-temp-file \"figure\" \".png\") :exports both \nimport matplotlib.pyplot as plt\n\nimport numpy\nx=numpy.linspace(-15,15)\nplt.figure(figsize=(10,5))\nplt.plot(x,numpy.cos(x)/x)\nplt.tight_layout()\n\nplt.savefig(matplot_lib_filename)\nreturn(matplot_lib_filename)\n#+end_src" "<src lang=\"python\">\n\n</src>"))
(add-to-list 'org-structure-template-alist
'("b" "#+begin_src shell :results output :exports both\n\n#+end_src" "<src lang=\"sh\">\n\n</src>"))
(add-to-list 'org-structure-template-alist
'("B" "#+begin_src shell :session *shell* :results output :exports both \n\n#+end_src" "<src lang=\"sh\">\n\n</src>"))
(add-to-list 'org-structure-template-alist
'("g" "#+begin_src dot :results output graphics :file \"/tmp/graph.pdf\" :exports both
digraph G {
node [color=black,fillcolor=white,shape=rectangle,style=filled,fontname=\"Helvetica\"];
A[label=\"A\"]
B[label=\"B\"]
A->B
}\n#+end_src" "<src lang=\"dot\">\n\n</src>"))
(add-hook 'org-babel-after-execute-hook 'org-display-inline-images)
(add-hook 'org-mode-hook 'org-display-inline-images)
(add-hook 'org-mode-hook 'org-babel-result-hide-all)
# -*- mode: org -*-
#+TITLE: Emacs init file written in org-mode
#+AUTHOR: Arnaud Legrand
#+EMAIL: arnaud.legrand@imag.fr
#+STARTUP: indent
#+LANGUAGE: en
This is a "minimal" =.emacs= configuration file. For this configuration
to work, you'll need:
- emacs25 (>25.2) (this way installing through elpa will save you some
time) and emacs-goodies (e.g., for color themes)
- org-mode > 9.0 (there has been several major compatibility breakage
so need to stay away from old versions)
- ess and R
- texlive and auctex
* Installation Instructions
Backup you own =.emacs= if you have one and replace it with [[file:init.el][the content
of this file]]. If you ever want to regenerate if from this org
document, simply =M-x org-babel-tangle=.
* Loading files:
** Loading babel: :ARNAUD:
#+begin_src emacs-lisp :tangle init.el
(require 'org-install)
(require 'org)
#+end_src
* Emacs aesthetics:
** Remove splash screen: :WEB:
#+begin_src emacs-lisp :tangle init.el
(setq inhibit-splash-screen t)
#+end_src
** Change frame name of emacs: :LUKA:WEB:
#+begin_src emacs-lisp :tangle init.el
(setq frame-title-format
'("Emacs - " (buffer-file-name "%f"
(dired-directory dired-directory "%b"))))
#+end_src
** Text coloring :ARNAUD:
#+begin_src emacs-lisp :tangle init.el
(global-font-lock-mode t)
(custom-set-faces
'(flyspell-incorrect ((t (:inverse-video t)))))
#+end_src
** Line and column numbering :ARNAUD:
#+begin_src emacs-lisp :tangle init.el
(line-number-mode 1)
(column-number-mode 1)
#+end_src
** Parenthesis :ARNAUD:
#+begin_src emacs-lisp :tangle init.el
(load-library "paren")
(show-paren-mode 1)
(transient-mark-mode t)
(require 'paren)
#+end_src
** Asking for confirmation concisely: :WEB:
Link: http://org.ryuslash.org/dotfiles/emacs/init.html#sec-7-1 Being
asked to type in yes explicitly all the time gets very tedious. I
understand that it is safer since y is much easier to type in
accidentally than yes and so the potential to say yes to things you
don't want is there, but I haven't had any such problems yet.
#+begin_src emacs-lisp :tangle init.el
(defalias 'yes-or-no-p 'y-or-n-p)
#+end_src
* Shortcuts:
** UTF 8 by default :ARNAUD:
#+begin_src emacs-lisp :tangle init.el
(set-terminal-coding-system 'utf-8)
(set-keyboard-coding-system 'utf-8)
(prefer-coding-system 'utf-8)
#+end_src
** Keyboard fix for Mac OS X users... :ADRIEN:
Thanks to Adrien Lebre for providing me with this
=ns-use-mac-modifier-symbols= thing. Unfortunately, this is broken at
the moment (I tried to wrap the command around the cond but it should
fail on mac).
#+begin_src emacs-lisp :tangle init.el
(setq
ns-command-modifier 'meta ; Apple/Command key is Meta
ns-alternate-modifier nil ; Option is the Mac Option key
ns-use-mac-modifier-symbols nil ; display standard Emacs (and not standard Mac) modifier symbols
)
#+end_src
** CUA mode a.k.a =C-x, C-c, C-v=
Yuck! Many people like this but I hate it as it conflicts with some of
my other shortcuts (e.g., the ones for spelling a region).
#+begin_src emacs-lisp :tangle init.el
(cua-mode t)
#+end_src
** Navigate back in text :ARNAUD:
#+begin_src emacs-lisp :tangle init.el
(defun jump-mark ()
(interactive)
(set-mark-command (point)))
(defun beginning-of-defun-and-mark ()
(interactive)
(push-mark (point))
(beginning-of-defun))
(defun end-of-defun-and-mark ()
(interactive)
(push-mark (point))
(end-of-defun))
(global-set-key "\^c\^b" 'beginning-of-defun-and-mark)
(global-set-key "\^c\^e" 'end-of-defun-and-mark)
(global-set-key "\^c\^j" 'jump-mark)
(global-set-key [S-f6] 'jump-mark) ;; jump from mark to mark
#+end_src
** Goto line :ARNAUD:
#+begin_src emacs-lisp :tangle init.el
(global-set-key "\M-g" 'goto-line)
#+end_src
** Increase/decrease text size in emacs :WEB:
#+begin_src emacs-lisp :tangle init.el
(global-set-key (kbd "C-+") 'text-scale-increase)
(global-set-key (kbd "C--") 'text-scale-decrease)
;; C-x C-0 restores the default font size
#+end_src
[11:26:47; 18.04.2014]
** Add spell checker to the file :WEB:
#+begin_src emacs-lisp :tangle init.el
;; Inspired from http://tex.stackexchange.com/questions/166681/changing-language-of-flyspell-emacs-with-a-shortcut
;; (defun spell (choice)
;; "Switch between language dictionaries."
;; (interactive "cChoose: (a) American | (f) Francais")
;; (cond ((eq choice ?1)
;; (setq flyspell-default-dictionary "american")
;; (setq ispell-dictionary "american")
;; (ispell-kill-ispell))
;; ((eq choice ?2)
;; (setq flyspell-default-dictionary "francais")
;; (setq ispell-dictionary "francais")
;; (ispell-kill-ispell))
;; (t (message "No changes have been made."))) )
(define-key global-map (kbd "C-c s a") (lambda () (interactive) (ispell-change-dictionary "american")))
(define-key global-map (kbd "C-c s f") (lambda () (interactive) (ispell-change-dictionary "francais")))
(define-key global-map (kbd "C-c s r") 'flyspell-region)
(define-key global-map (kbd "C-c s b") 'flyspell-buffer)
(define-key global-map (kbd "C-c s s") 'flyspell-mode)
#+end_src
** Reload buffer :ARNAUD:
#+begin_src emacs-lisp :tangle init.el
(global-set-key [f5] '(lambda () (interactive) (revert-buffer nil t nil)))
#+end_src
* Small fixes for LaTeX:
** PDF with LaTeX by default :ARNAUD:
#+begin_src emacs-lisp :tangle init.el
(defun auto-fill-mode-on () (TeX-PDF-mode 1))
(add-hook 'tex-mode-hook 'TeX-PDF-mode-on)
(add-hook 'latex-mode-hook 'TeX-PDF-mode-on)
(setq TeX-PDF-mode t)
#+end_src
** Auto-fill-mode
#+begin_src emacs-lisp :tangle init.el
(defun auto-fill-mode-on () (auto-fill-mode 1))
(add-hook 'text-mode-hook 'auto-fill-mode-on)
(add-hook 'emacs-lisp-mode 'auto-fill-mode-on)
(add-hook 'tex-mode-hook 'auto-fill-mode-on)
(add-hook 'latex-mode-hook 'auto-fill-mode-on)
#+end_src
* Org-mode convenient configuration
** Default directory
#+begin_src emacs-lisp :tangle init.el
(setq org-directory "~/org/")
#+end_src
** Cosmetics
#+begin_src emacs-lisp :tangle init.el
(setq org-hide-leading-stars t)
(setq org-alphabetical-lists t)
(setq org-src-fontify-natively t) ;; you want this to activate coloring in blocks
(setq org-src-tab-acts-natively t) ;; you want this to have completion in blocks
(setq org-hide-emphasis-markers t) ;; to hide the *,=, or / markers
(setq org-pretty-entities t) ;; to have \alpha, \to and others display as utf8 http://orgmode.org/manual/Special-symbols.html
#+end_src
** Agenda
*** Add short cut keys for the org-agenda :ARNAUD:
#+begin_src emacs-lisp :tangle init.el
(global-set-key "\C-cl" 'org-store-link)
(global-set-key "\C-cc" 'org-capture)
(global-set-key (kbd "C-c a") 'org-agenda)
(define-key global-map "\C-cl" 'org-store-link)
(define-key global-map (kbd "C-c a") 'org-agenda)
(global-set-key "\C-cb" 'org-iswitchb)
(setq org-default-notes-file "~/org/notes.org")
(define-key global-map "\C-cd" 'org-capture)
(setq org-capture-templates (quote (("t" "Todo" entry (file+headline "~/org/liste.org" "Tasks") "* TODO %?
%i
%a" :prepend t) ("j" "Journal" entry (file+datetree "~/org/journal.org") "* %?
Entered on %U
%i
%a"))))
#+end_src
#+RESULTS:
| t | Todo | entry | (file+headline ~/org/liste.org Tasks) | * TODO %?\n %i\n %a | :prepend | t |
| j | Journal | entry | (file+datetree ~/org/journal.org) | * %?\nEntered on %U\n %i\n %a | | |
*** Agenda config :ARNAUD:
#+begin_src emacs-lisp :tangle init.el
(setq org-agenda-include-all-todo t)
(setq org-agenda-include-diary t)
#+end_src
* Org-mode shortcuts
** Adding date with brackets with command "C-c d": :LUKA:
#+begin_src emacs-lisp :tangle init.el
(global-set-key (kbd "C-c d") 'insert-date)
(defun insert-date (prefix)
"Insert the current date. With prefix-argument, use ISO format. With
two prefix arguments, write out the day and month name."
(interactive "P")
(let ((format (cond
((not prefix) "** %Y-%m-%d")
((equal prefix '(4)) "[%Y-%m-%d]"))))
(insert (format-time-string format))))
#+end_src
[16:34:01; 24.04.2013]
** Adding date with command "C-c t": :LUKA:
#+begin_src emacs-lisp :tangle init.el
(global-set-key (kbd "C-c t") 'insert-time-date)
(defun insert-time-date (prefix)
"Insert the current date. With prefix-argument, use ISO format. With
two prefix arguments, write out the day and month name."
(interactive "P")
(let ((format (cond
((not prefix) "[%H:%M:%S; %d.%m.%Y]")
((equal prefix '(4)) "[%H:%M:%S; %Y-%m-%d]"))))
(insert (format-time-string format))))
#+end_src
[16:34:15; 24.04.2013]
** Org-store-link: :ARNAUD:
#+begin_src emacs-lisp :tangle init.el
(global-set-key (kbd "C-c l") 'org-store-link)
#+end_src
** Navigating through org-mode:
Additional shortcuts for navigating through org-mode documents:
#+begin_src emacs-lisp :tangle init.el
(global-set-key (kbd "C-c <up>") 'outline-up-heading)
(global-set-key (kbd "C-c <left>") 'outline-previous-visible-heading)
(global-set-key (kbd "C-c <right>") 'outline-next-visible-heading)
#+end_src
* Org-mode + babel:
** Seamless use of babel (no confirmation, lazy export) :ARNAUD:
#+begin_src emacs-lisp :tangle init.el
;; In org-mode 9 you need to have #+PROPERTY: header-args :eval never-export
;; in the beginning or your document to tell org-mode not to evaluate every
;; code block every time you export.
(setq org-confirm-babel-evaluate nil) ;; Do not ask for confirmation all the time!!
#+end_src
** Some initial languages we want org-babel to support: :ARNAUD:
#+begin_src emacs-lisp :tangle init.el
(org-babel-do-load-languages
'org-babel-load-languages
'(
(shell . t)
(python . t)
(R . t)
(ruby . t)
(ocaml . t)
(ditaa . t)
(dot . t)
(octave . t)
(sqlite . t)
(perl . t)
(screen . t)
(plantuml . t)
(lilypond . t)
(org . t)
(makefile . t)
))
(setq org-src-preserve-indentation t)
#+end_src
** Adding source code blocks: :LUKA:
*** With capital letters:
To use this type <s and then TAB
#+begin_src emacs-lisp :tangle init.el
(add-to-list 'org-structure-template-alist
'("s" "#+begin_src ?\n\n#+end_src" "<src lang=\"?\">\n\n</src>"))
#+end_src
*** Emacs-elisp code:
To use this type <m and then TAB
#+begin_src emacs-lisp :tangle init.el
(add-to-list 'org-structure-template-alist
'("m" "#+begin_src emacs-lisp :tangle init.el\n\n#+end_src" "<src lang=\"emacs-lisp\">\n\n</src>"))
#+end_src
*** R code:
To use this type <r and then TAB. This creates an R block for textual output.
#+begin_src emacs-lisp :tangle init.el
(add-to-list 'org-structure-template-alist
'("r" "#+begin_src R :results output :session *R* :exports both\n\n#+end_src" "<src lang=\"R\">\n\n</src>"))
#+end_src
To use this type <R and then TAB. This creates an R block for graphics
that are stored in the =/tmp/=.
#+begin_src emacs-lisp :tangle init.el
(add-to-list 'org-structure-template-alist
'("R" "#+begin_src R :results output graphics :file (org-babel-temp-file \"figure\" \".png\") :exports both :width 600 :height 400 :session *R* \n\n#+end_src" "<src lang=\"R\">\n\n</src>"))
#+end_src
To use this type <RR and then TAB. This creates an R block for
graphics that are stored in the directory of the current file.
#+begin_src emacs-lisp :tangle init.el
(add-to-list 'org-structure-template-alist
'("RR" "#+begin_src R :results output graphics :file (org-babel-temp-file (concat (file-name-directory (or load-file-name buffer-file-name)) \"figure-\") \".png\") :exports both :width 600 :height 400 :session *R* \n\n#+end_src" "<src lang=\"R\">\n\n</src>"))
#+end_src
*** Python code
To use this type <p and then TAB
#+begin_src emacs-lisp :tangle init.el
(add-to-list 'org-structure-template-alist
'("p" "#+begin_src python :results output :exports both\n\n#+end_src" "<src lang=\"python\">\n\n</src>"))
#+end_src
#+begin_src emacs-lisp :tangle init.el
(add-to-list 'org-structure-template-alist
'("P" "#+begin_src python :results output :session *python* :exports both\n\n#+end_src" "<src lang=\"python\">\n\n</src>"))
#+end_src
#+begin_src emacs-lisp :tangle init.el
(add-to-list 'org-structure-template-alist
'("PP" "#+begin_src python :results file :var matplot_lib_filename=(org-babel-temp-file \"figure\" \".png\") :exports both \nimport matplotlib.pyplot as plt\n\nimport numpy\nx=numpy.linspace(-15,15)\nplt.figure(figsize=(10,5))\nplt.plot(x,numpy.cos(x)/x)\nplt.tight_layout()\n\nplt.savefig(matplot_lib_filename)\nreturn(matplot_lib_filename)\n#+end_src" "<src lang=\"python\">\n\n</src>"))
#+end_src
*** Bash "sh" code:
To use this type <b and then TAB
#+begin_src emacs-lisp :tangle init.el
(add-to-list 'org-structure-template-alist
'("b" "#+begin_src shell :results output :exports both\n\n#+end_src" "<src lang=\"sh\">\n\n</src>"))
#+end_src
To use this type <B and then TAB. This comes with a session argument
(e.g., in case you want to keep ssh connexions open).
#+begin_src emacs-lisp :tangle init.el
(add-to-list 'org-structure-template-alist
'("B" "#+begin_src shell :session *shell* :results output :exports both \n\n#+end_src" "<src lang=\"sh\">\n\n</src>"))
#+end_src
*** Graphviz
#+begin_src emacs-lisp :tangle init.el
(add-to-list 'org-structure-template-alist
'("g" "#+begin_src dot :results output graphics :file \"/tmp/graph.pdf\" :exports both
digraph G {
node [color=black,fillcolor=white,shape=rectangle,style=filled,fontname=\"Helvetica\"];
A[label=\"A\"]
B[label=\"B\"]
A->B
}\n#+end_src" "<src lang=\"dot\">\n\n</src>"))
#+end_src
** Evaluating whole subtree: :LUKA:
#+begin_src emacs-lisp
(global-set-key (kbd "C-c S-t") 'org-babel-execute-subtree)
#+end_src
** Display images :ARNAUD:
#+begin_src emacs-lisp :tangle init.el
(add-hook 'org-babel-after-execute-hook 'org-display-inline-images)
(add-hook 'org-mode-hook 'org-display-inline-images)
(add-hook 'org-mode-hook 'org-babel-result-hide-all)
#+end_src
# -*- mode: org -*-
# -*- coding: utf-8 -*-
#+STARTUP: overview indent inlineimages logdrawer
#+TITLE: Journal
#+AUTHOR: Put your name here
#+LANGUAGE: en
#+TAGS: LIG(L) HOME(H) Europe(E) Blog(B) noexport(n) Stats(S)
#+TAGS: Epistemology(E) Vulgarization(V) Teaching(T) R(R) OrgMode(O) Python(P)
#+EXPORT_SELECT_TAGS: Blog
#+OPTIONS: H:3 num:t toc:t \n:nil @:t ::t |:t ^:t -:t f:t *:t <:t
#+OPTIONS: TeX:t LaTeX:nil skip:nil d:nil todo:t pri:nil tags:not-in-toc
#+EXPORT_SELECT_TAGS: export
#+EXPORT_EXCLUDE_TAGS: noexport
#+COLUMNS: %25ITEM %TODO %3PRIORITY %TAGS
#+SEQ_TODO: TODO(t!) STARTED(s!) WAITING(w@) APPT(a!) | DONE(d!) CANCELLED(c!) DEFERRED(f!)
* 2018
** 2018-02 March
*** 2018-02-12 Monday
**** Demonstrating Emacs/Orgmode shortcuts
These informations were gathered and first demonstrated in my [[https://github.com/alegrand/RR_webinars/blob/master/1_replicable_article_laboratory_notebook/index.org][First
webinar on reproducible research: litterate programming]].
***** Emacs shortcuts
Here are a few convenient emacs shortcuts for those that have never
used emacs. In all of the emacs shortcuts, =C=Ctrl=, =M=Alt/Esc= and
=S=Shift=. Note that you may want to use two hours to follow the emacs
tutorial (=C-h t=). In the configuration file CUA keys have been
activated and allow you to use classical copy/paste (=C-c/C-v=)
shortcuts. This can be changed from the Options menu.
- =C-x C-c= exit
- =C-x C-s= save buffer
- =C-g= panic mode ;) type this whenever you want to exit an awful
series of shortcuts
- =C-Space= start selection marker although selection with shift and
arrows should work as well
- =C-l= reposition the screen
- =C-_= (or =C-z= if CUA keys have been activated)
- =C-s= search
- =M-%= replace
- =C-x C-h= get the list of emacs shortcuts
- =C-c C-h= get the list of emacs shortcuts considering the mode you are
currently using (e.g., C, Lisp, org, ...)
There are a bunch of cheatsheets also available out there (e.g.,
[[http://www.shortcutworld.com/en/linux/Emacs_23.2.1.html][this one for emacs]] and [[http://orgmode.org/orgcard.txt][this one for org-mode]] or this [[http://sachachua.com/blog/wp-content/uploads/2013/05/How-to-Learn-Emacs-v2-Large.png][graphical one]]).
***** Org-mode
Many emacs shortcuts start by =C-x=. Org-mode's shortcuts generaly
start with =C-c=.
- =Tab= fold/unfold
- =C-c c= capture (finish capturing with =C-c C-c=, this is explained on
the top of the buffer that just opened)
- =C-c C-c= do something useful here (tag, execute, ...)
- =C-c C-o= open link
- =C-c C-t= switch todo
- =C-c C-e= export
- =M-Enter= new item/section
- =C-c a= agenda (try the =L= option)
- =C-c C-a= attach files
- =C-c C-d= set a deadl1ine (use =S-arrows= to navigate in the dates)
- =A-arrows= move subtree (add shift for the whole subtree)
***** Org-mode Babel (for literate programming)
- =<s + tab= template for source bloc. You can easily adapt it to get
this:
#+BEGIN_EXAMPLE
#+begin_src shell
ls
#+end_src
#+END_EXAMPLE
Now if you =C-c C-c=, it will execute the block.
#+BEGIN_EXAMPLE
#+RESULTS:
| #journal.org# |
| journal.html |
| journal.org |
| journal.org~ |
#+END_EXAMPLE
- Source blocks have many options (formatting, arguments, names,
sessions,...), which is why I have my own shortcuts =<b + tab= bash
block (or =B= for sessions).
#+BEGIN_EXAMPLE
#+begin_src shell :results output :exports both
ls /tmp/*201*.pdf
#+end_src
#+RESULTS:
: /tmp/2015_02_bordeaux_otl_tutorial.pdf
: /tmp/2015-ASPLOS.pdf
: /tmp/2015-Europar-Threadmap.pdf
: /tmp/europar2016-1.pdf
: /tmp/europar2016.pdf
: /tmp/M2-PDES-planning-examens-janvier2016.pdf
#+END_EXAMPLE
- I have defined many such templates in my configuration. You can
give a try to =<r=, =<R=, =<RR=, =<g=, =<p=, =<P=, =<m= ...
- Some of these templates are not specific to babel: e.g., =<h=, =<l=,
=<L=, =<c=, =<e=, ...
# -*- mode: org -*-
#+TITLE: Rstudio
#+AUTHOR: Arnaud Legrand
#+DATE: June, 2018
#+STARTUP: overview indent
* Table of Contents :TOC:
- [[#installing-rstudio][Installing Rstudio]]
- [[#linux-debian-ubuntu][Linux (debian, ubuntu)]]
- [[#mac-osx-and-windows][Mac OSX and Windows]]
- [[#installing-git][Installing Git]]
- [[#linux-debian-ubuntu][Linux (debian, ubuntu)]]
- [[#mac-osx-and-windows][Mac OSX and Windows]]
- [[#rstudio-and-gitgithubgitlab][Rstudio and git/github/gitlab]]
- [[#cheat-sheets][Cheat Sheets]]
* Installing Rstudio
** Linux (debian, ubuntu)
We provide here only instructions for debian-based distributions. Feel
free to contribute to this document to provide up-to-date information
for other distributions (e.g.n redhat, fedora).
Today, the stable versions of the most common distributions provide
recent enough versions of emacs and org-mode:
- Debian (stretch) ships with [[https://packages.debian.org/stretch/r-base][R 3.3.3-1]], [[https://packages.debian.org/stretch/r-cran-knitr][knitr 1.15.1]], and [[https://packages.debian.org/stretch/r-cran-ggplot2][ggplot 2.2.1]]
- Ubuntu (bionic 18.04) ships with [[https://packages.ubuntu.com/bionic/r-base][R 3.4.4]], and [[https://packages.ubuntu.com/bionic/r-cran-knitr][knitr 1.17]], and [[https://packages.ubuntu.com/bionic/r-cran-ggplot2][ggplot 2.2.1]]
- Ubuntu (artful 17.04) ships with [[https://packages.ubuntu.com/artful/r-base][R 3.4.2]], and [[https://packages.ubuntu.com/artful/r-cran-knitr][knitr 1.15]], and [[https://packages.ubuntu.com/artful/r-cran-ggplot2][ggplot 2.2.1]]
If your distribution is older than this, well, it may be a good time
for upgrading...
*** Installing R
Beforehand, you need to install the R language and convenient packages
by running (as root):
#+BEGIN_SRC shell
apt-get update ; sudo apt-get install r-base r-cran-knitr r-cran-ggplot2
#+END_SRC
Alternatively, if the installation of =r-cran-gplot2= or =r-cran-knitr=
fails, you may want to install them locally (through the R packaging
system) and manually by running the following commands in R (or
Rstudio):
#+BEGIN_SRC R
install.packages("knitr")
install.packages("ggplot2")
#+END_SRC
If you plan to export pdf documents with LaTeX, you probably also want
to run (as root):
#+begin_src sh :results output :exports both
apt-get update ; apt-get install texlive-base
#+end_src
*** Installing rstudio
Rstudio is unfortunately not packaged within debian so the easiest is
to download the corresponding debian package on the [[https://www.rstudio.com/products/rstudio/download/#download][Rstudio webpage]]
and then to install it manually (depending on when you do this, you
can obviously change the version number). Here is how to install it:
#+BEGIN_SRC shell
cd /tmp/
wget https://download1.rstudio.org/rstudio-xenial-1.1.453-amd64.deb
sudo dpkg -i rstudio-xenial-1.1.453-amd64.deb
sudo apt-get update ; sudo apt-get -f install # to fix possibly missing dependencies
#+END_SRC
** Mac OSX and Windows
#+BEGIN_QUOTE
Some instructions on installing R and knitr must be missing. This
should be tested and improved.
#+END_QUOTE
Download and install rstudio from the [[https://www.rstudio.com/products/rstudio/download/#download][Rstudio webpage]] by choosing the
right operating system.
Then open Rstudio and type the following commands in the console to
install knitr and ggplot2:
#+BEGIN_SRC R
install.packages("knitr")
install.packages("ggplot2")
#+END_SRC
* Installing Git
** Linux (debian, ubuntu)
We provide here only instructions for debian-based distributions. Feel
free to contribute to this document to provide up-to-date information
for other distributions (e.g.n redhat, fedora).
Run (as root):
#+begin_src sh :results output :exports both
apt-get update ; apt-get install git
#+end_src
** Mac OSX and Windows
- Download and install Git from the [[https://git-scm.com/downloads][Git website]].
- Optional Git clients (should not be needed if you work within
Rstudio):
- [[https://www.sourcetreeapp.com/][SourceTree]]
- [[github desktop][GitHub Desktop]]
#+BEGIN_QUOTE
[[https://github.com/desktop/desktop/issues/852][Apparently]], this works with gitlab and https.
#+END_QUOTE
* Rstudio and git/github/gitlab
#+BEGIN_QUOTE
We may want to redo a screencast in French for this
#+END_QUOTE
- [[https://www.rstudio.com/resources/webinars/rstudio-essentials-webinar-series-managing-part-2/][This video on using git/githugb/gitlab from Rstudio]] (starting at time 9:09)
Alternatively, we can point to:
http://www.geo.uzh.ch/microsite/reproducible_research/post/rr-rstudio-git/
- Part 1/item 2: Setup Git in RStudio
- Part 2 (to adapt to gitlab)
- Part 3
* Cheat Sheets
The Rstudio team has created a lot of good material and tutorials. You
should definitively look at the [[https://www.rstudio.com/resources/cheatsheets/][Cheat sheets webpage]]. In particular
you want to have look at the ones on:
- [[https://github.com/rstudio/cheatsheets/raw/master/rstudio-ide.pdf][the Rstudio IDE]],
- [[https://github.com/rstudio/cheatsheets/raw/master/rmarkdown-2.0.pdf][R Markdown]],
- the [[https://www.rstudio.com/wp-content/uploads/2015/03/rmarkdown-reference.pdf][R Markdown Reference guide]],
- [[https://github.com/rstudio/cheatsheets/raw/master/data-visualization-2.1.pdf][data visualization with ggplot2]],
- [[https://github.com/rstudio/cheatsheets/raw/master/data-transformation.pdf][data transformation with dplyr]]
In case it helps, here are some (sometimes outdated) French versions
of these documents:
- [[https://github.com/rstudio/cheatsheets/raw/master/translations/french/rstudio-IDE-cheatsheet.pdf][L'IDE Rstudio]]
- [[https://github.com/rstudio/cheatsheets/raw/master/translations/french/ggplot2-french-cheatsheet.pdf][Visualisation de données avec ggplot2]]
- [[https://github.com/rstudio/cheatsheets/raw/master/translations/french/data-wrangling-french.pdf][Transformation de données avec dplyr]]
- [[https://www.fun-mooc.fr/c4x/UPSUD/42001S02/asset/RMarkdown.pdf][Un court document sur R Markdown]]
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment