From 1d9b100e57f8af553f609eded8e21ea45ab9c664 Mon Sep 17 00:00:00 2001 From: Arnaud Legrand Date: Wed, 5 Sep 2018 11:27:21 +0200 Subject: [PATCH] Automatically generated HTML files to forward in GitLab --- module2/ressources/gitlab.html | 361 +++++++++++++++++++++++--------- module2/ressources/jupyter.html | 149 +++++++++---- 2 files changed, 369 insertions(+), 141 deletions(-) diff --git a/module2/ressources/gitlab.html b/module2/ressources/gitlab.html index 666d7c9..955144d 100644 --- a/module2/ressources/gitlab.html +++ b/module2/ressources/gitlab.html @@ -4,26 +4,27 @@

Table of Contents

@@ -36,15 +37,9 @@ GitLab in the context of this MOOC.

So far, you only used git via the web interface from the GitLab we -deployed for the MOOC. +deployed for the MOOC: https://app-learninglab.inria.fr/gitlab/

-
-

-https://app-learninglab.inria.fr/gitlab/ -

-
-

If you access this link from the FUN platform, you do not have to authenticate and you can readily read and modify all your files. This @@ -66,13 +61,13 @@ Please read all these instructions carefully, in particular the one on "Configuring your password on GitLab".

-
-

Installing Git

-
+
+

Installing Git

+
-
-

Linux (debian, ubuntu)

-
+
+

Linux (debian, ubuntu)

+

We provide here only instructions for debian-based distributions. Feel free to contribute to this document to provide up-to-date information @@ -88,9 +83,9 @@ Run (as root):

-
-

Mac OSX and Windows

-
+
+

Mac OSX and Windows

+
  • Download and install Git from the Git website.
  • Optional Git clients (should not be needed if you work within @@ -110,13 +105,13 @@ Rstudio):
-
-

Configuring Git

-
+
+

Configuring Git

+
-
-

Telling Git who you are: Name and Email

-
+
+

Telling Git who you are: Name and Email

+
  1. Open Terminal.

  2. @@ -145,13 +140,13 @@ email@example.com

-
-

Dealing with proxies

-
+
+

Dealing with proxies

+

You may be behind a proxy, in which case you may have trouble cloning or fetching from a remote repository or you may get an error like -unable to access '...' Couldn't resolve host '...'. +unable to access ... Couldn't resolve host ...

@@ -163,90 +158,95 @@ In such case, consider something like this:

-The proxyPassword will be stored unencrypted in your .gitconfig file, +The proxyPassword will be stored in plain text (unencrypted) in your .gitconfig file, which you may not want. In that case, remove it from the URL and you will be prompted for it every time it is needed.

-
-

Configuring your password on GitLab

-
-
    -

  1. -Access GitLab. Note: You have to access Gitlab from the FUN -platform otherwise you may get a 405 error when trying to direcly -open https://app-learninglab.inria.fr/gitlab/users/sign_in. +

    +

    Getting your default password on GitLab (and possibly changing it)

    +
    +

    +Warning (Jupyter users) : changing your default Gitlab password will prevent you from committing in Jupyter. You will have to do the extra step of changing your Jupyter ~/.git-credentials

    -
    -
    -

    erreur405.png

    -
    -
    2. +

    1. -Click on Sign in: +Get your default password using the Gitlab credentials retrieval +tool as described on the corresponding resource.

      -

      projects.png

      +

      password_retrieval.png

      -The long and uggly character sequence, which is here replaced by xxx -is your GitLab login/id. You'll need to use it when trying to -propagate some modifications from your computer to GitLab. -

      2. -

    2. -To defind your password, you should go in settings: +The first long and ugly character sequence is your GitLab +login/id. It is easy to find once you are logged on gitlab. The +second one however is your password and this webpage is the only +place where you can find it. We used the FUN authentification +mechanism to propagate your credentials so only you can have access +to it. You'll need to use this password when trying to propagate +some modifications from your computer to GitLab. +

      + +

      +Note: You have to access this webpage from the FUN platform +otherwise you may get a 405 error when trying to direcly open +https://app-learninglab.inria.fr/jupyterhub/services/password.

      -

      settings.png

      +

      erreur405.png

    3. -Then, go to the profile tab to define a valid email address: +Access GitLab.

      -
      -
      -

      profile.png

      -
      -

      -You will then receive a email with a link to open to confirm your -email address is correct. Just open it. +Note: Again, you have to access Gitlab from the FUN platform +otherwise you may get a 405 error when trying to direcly open +https://app-learninglab.inria.fr/gitlab/users/sign_in.

    4. -Once your email address is validated, it will appear as verified and -as your main address in the Emails tab: +Click on the first Sign in button (alternatively, you can the +login/password you just retrieved and use the second Sign in +button).

      -

      mail.png

      +

      signin.png

    5. -You will then finally be able to click on the I forgot my password -button from the password tab: -password.png

      +If you wish to modify your password, you shoul go to Account > + Settings > Password and define your password using the default +password you just retrieved. Again, if you use the Jupyter +notebooks we have deployed for the MOOC, remember that changing +your default Gitlab password will prevent you from committing in +Jupyter. You will have to do the extra step of changing your +Jupyter ~/.git-credentials through a jupyter console (see next +section). +

      +
      -

      -You will then receive an email allowing you to reset your GitLab -password. -

      6. +
      +

      password.png

      +
      +
-
-

Remembering your password locally

-
+
+

Remembering your password locally

+

-If you clone your repos by simply pasting the GitLab URL, you will be +If you clone your repository by simply pasting the GitLab URL, you will be prompted for your login and your password every time you want to propagate your local modifications, which is tedious. This is why you should ask git to remember your login and password as follows @@ -262,11 +262,25 @@ cached in memory and they will not be asked again before an hour. You may want to read these instructions to better understand how all this works.

+ +

+If you want your password to be permanently remembered, you should use +this command +

+
+
git config credential.helper store
+
+
+

+Your password will be then stored in a .git-credentials file in plain +text. On a perfectly secured machine, it may be fine… or not… ;) +Use it at your own risk. +

-
-

Optional: authenticating through SSH

-
+
+

Optional: authenticating through SSH

+

There are two ways of authenticating and synchronizing your local repository with GitLab: through HTTPS or through SSH. The first one is @@ -279,13 +293,13 @@ your SSH public key on GitLab. This section provides with information on how to do this.

-
-

Installing SSH

-
+
+

Installing SSH

+
    -
  • Linux (debian, ubuntu)
    -
    +
  • Linux (debian, ubuntu)
    +

    We provide here only instructions for debian-based distributions. Feel free to contribute to this document to provide up-to-date information @@ -301,15 +315,15 @@ Run (as root):

-
  • Mac OSX
    -
    +
  • Mac OSX
    +

    You do not have anything to do as it is installed by default.

    • -
  • Windows
    -
    +
  • Windows
    +

    You should install the Putty client. Once it is installed, look for the section on generating an SSH key. @@ -318,9 +332,9 @@ the section on generati

    • -
    -

    Setting up SSH on GitLab

    -
    +
    +

    Setting up SSH on GitLab

    +

    Here are all the official explanations on how to set up your SSH key on GitLab. Alternatively, you may also want to have a look at this @@ -331,4 +345,157 @@ video:

    +
    +

    Using Git through the command line to synchronize your local files with Gitlab

    +
    +

    +This section describes a generic (through the command line) way to +synchronize your local files with Gitlab. You will not need this if +you follow the Jupyter course. If you follow the Rstudio course, all +these operations can be done through Rstudio and you may want to read +the corresponding instructions. If you follow the Org-Mode course, all +these operations can be done through Magit and you may want to read +the corresponding instructions. +

    + +

    +Here are other ways to learn Git through the command line: +

    + + +

    +Now, let's start! +

    +
      +
    1. Obtain the repository URL +adresse_depot.png

    2. +Cloning the repository +

      +
      +
      cd /the/directory/where/you/want/to/clone/your/repository
+git clone https://app-learninglab.inria.fr/gitlab/xxx/mooc-rr.git
+
      +
      +

      +Alternatively, you may want to indicate now your login although I +rather suggest you follow the Remembering your password locally +instructions: +

      +
      +
      git clone https://xxx@app-learninglab.inria.fr/gitlab/xxx/mooc-rr.git
+
      +
      +

      +Now a directory mooc-rr has been created on your computer. +

      3. +

    3. +Inspect the repository +

      +
      +
      cd mooc-rr
+ls # (Unix)
+dir # (Windows)
+
      +
      4. +

    4. +Synchronizing to GitLab +

      + +

      +You should indicate which files to track (git add) and commit them +locally (git commit) before they can be transfered (git push) to +GitLab. The git status will indicate you whether files are +tracked/modified/committed/… +

      + +

      +Let's assume you just created a `fichier.txt` file on the top of +the mooc-rr directory. +

      + +
      +
      git status
+
      +
      + + +
      +

      status1.png

      +
      + +
      +
      git add fichier.txt
+git status
+
      +
      + + +
      +

      status2.png

      +
      + +
      +
      git commit -m "message commit"
+
      +
      + + +
      +

      commit_git.png

      +
      + +
      +
      git status
+
      +
      + + +
      +

      status3.png

      +
      + +

      +The file can then be transfered to GitLab: +

      +
      +
      git push
+
      +
      + +

      +At this point, git will as you about your login/password unless +you followed the previous Remembering your password locally instructions. +

      + +

      +N.B.: you will not be allowed to propagate your modifications to +GitLab if other modifications (e.g., from someone else) have been +propagated in between +

      + + +
      +

      rejected.png

      +
      5. +

    5. +Synchronizing from Gitlab: to avoid the previous problem, you need +to fetch the remote GitLab modifications first and apply them +locally. +

      +
      +
      git pull
+
      +
      +

      +Only then will you be able to git push. +

      6. +
    +
    +
    diff --git a/module2/ressources/jupyter.html b/module2/ressources/jupyter.html index ae2b993..7406f80 100644 --- a/module2/ressources/jupyter.html +++ b/module2/ressources/jupyter.html @@ -4,43 +4,49 @@

    Table of Contents

    -
    -

    Additional Jupyter resources or tricks

    -
    +
    +

    Additional Jupyter resources or tricks

    +
    -
    -

    Tips and tricks

    -
    +
    +

    Tips and tricks

    +

    The following webpage lists several Jupyter tricks (in particular, it illustrates many Ipython magic commands) that should improve your efficiency (note that this blog post is about two years old so some of -the tricks may have been integrated in the default behavior of jupyter +the tricks may have been integrated in the default behavior of Jupyter now).

    -
    -

    Running R and Python in the same notebook

    -
    +
    +

    Running R and Python in the same notebook

    +

    The best solution to this is to install rpy2. On my machine, I have installed the python3-rpy2 debian package with apt-get install. E.g., @@ -88,9 +94,9 @@ plot(df)

    -
    -

    Exporting a notebook

    -
    +
    +

    Exporting a notebook

    +

    Obviously, you can convert to html or pdf using the using the File > Download as > HTML (or PDF) menu option. This can also be done from @@ -117,24 +123,29 @@ taste.

    -
    -

    Installing Jupyter on your own machine

    -
    +
    +

    Installing Jupyter on your own machine

    +
    -
    -

    Installing jupyter

    -
    +
    +

    Installing Jupyter

    +
    +

    +Follow these instructions if you wish to have a Jupyter environment on +your own machine similar to the one we set up for this MOOC. +

    +

    -Follow these instructions if you wish to have a similar Jupyter environment on you own machine. +First, download and install the latest version of Miniconda. We use +Miniconda version 4.5.4 and Python version 3.6 on our server.

    -First, download and install Miniconda latest version. We use Miniconda version 4.5.4 and Python version 3.6 on our server .
    -Miniconda is a light version of Anaconda which includes Python, the Jupyter Notebook, and other commonly used packages for scientific computing and data science. +Miniconda is a light version of Anaconda, which includes Python, the Jupyter Notebook, and other commonly used packages for scientific computing and data science.

    -Then download the moocrr environment file and create it using conda: +Then download the moocrr environment file and create the environment using conda: 

    conda env create -f environment.yml
@@ -150,31 +161,47 @@ jupyter notebook
    - -
    -

    Exporting your notebooks with latex

    -
    +
    +

    Side note about Jupyter, JupyterLab, JupyterHub, …

    +
    +

    +Note that Jupyter notebooks are only a small part of the picture and +that Jupyter is now part of a bigger project: JupyterLab, which allows +you to mix various components (including notebooks) in your +browser. In the context of this MOOC, our time frame was too short to +benefit from JupyterLab which was still under active development but +this is probably the best option now if you want to benefit from +cutting-edge Jupyter notebooks. +

    +
    +
    +
    +

    Exporting your notebooks with latex

    +

    -Here is what I had to install on my recent debian machine to make sure +Here is what we had to install on our recent debian machine to make sure the notebook export via latex works:

    -
    sudo apt-get install wkhtmltopdf
-sudo apt-get install texlive-xetex
+
    sudo apt-get install texlive-xetex wkhtmltopdf
    -
    -

    Interesting extensions to improve notebook readability

    -
    +
    +

    Jupyter extensions/plugins

    +
    +
    +
    +

    Improving notebook readability

    +

    -Here are two interesting extensions that can improve readability: +Here are a few extensions that can ease your life:

    • -Code folding +Code folding to improve readability when browsing the notebook. 

      pip3 install jupyter_contrib_nbextensions
@@ -182,7 +209,7 @@ Here are two interesting extensions that can improve readability:

    • -Hiding code +Hiding code to improve readability when exporting. 

      sudo pip3 install hide_code
@@ -194,5 +221,39 @@ jupyter-serverextension enable --py hide_code
    +
    +

    Interacting with GitLab and GitHub

    +
    +

    +To ease your experience, we added some pull/push buttons that allow +you to commit and sync with GitLab. This development was specific to +the MOOC but inspired from a previous proof of concept. We have +recently discovered that someone else developed about at the same time +a rather generic version of this Jupyter plugin. Otherwise, remember +that it is very easy to insert a shell cell in Jupyter in which you +can easily issue git commands. This is how we work most of the time. +

    + +

    +This being said, you may have noticed that Jupyter keeps a perfect +track of the sequence in which cells have been run by updating the +"output index". This is a very good property from the reproducibility +point of view but depending on your usage, you may find it a bit +painful when commiting. Some people have thus developped specific git +hooks to ignore these numbers when comitting Jupyter notebooks. There +is a long an interesting discussion about various options on +StackOverflow. +

    + +

    +Last but not least, remembter that Jupyter notebooks are only a small +part of the picture and that Jupyter is now part of a bigger project: +JupyterLab, which allows you to mix various components (including +notebooks) in your browser. A specific JupyterLab git plugin has been +developed to offer a nice version control experience. +

    +
    +
    +
    -- 2.18.1