it-artikel:linux:how-to-make-an-ubuntu-or-debian-apt-repository-locally-available-as-an-emergency-offline-fallback

How to make an Ubuntu or Debian apt repository locally available as an emergency offline fallback

Maybe you never had that problem yourself, but there may be reasons why someone want to have one or multiple Ubuntu or Debian online repositories available on your local network. Maybe because you throw an Linux installation party and dozens of times the same repos and files are been pulled over a thin DSL connection. Or you want your own local mirror in case your internet connection goes down. Or you might just want to keep a certain version or snapshot of the online repo for a specific UBUNTU release or distro, because one day it may become unavailable, if UBUNTU/DEBIAN decides to take it down if its “too old” for them. This might be interesting for some “retro computing” guys as well.

So how to achieve that ?

As always, there is more than just one solution. However… this is how i did it.

Pre-Requirements:

  1. Define what distro releases and what “repositories” or “sections” you want to have available on your LAN. With “repositories” or “sections” i mean those “keywords” that follow one of your “deb” lines in your /etc/apt/sources.list. In my example i want to have Ubuntu 16.04 LTS, Ubuntu 18.04 LTS and Ubuntu 19.04 available at all time. So ill need the code names for its respositories/sections later. The format is as shown here:
    #deb url distroReleaseName repo/sections
    deb http://de.archive.ubuntu.com/ubuntu/ disco main restricted universe multiverse
  2. Take a note what official Ubuntu/Debian mirror server your want to use for your download. For me it will be http://de.archive.ubuntu.com/ubuntu/
    • :!: BEWARE: This manual assumes the use of only ONE official mirror server, for all distros or releases we want! This is somewhat important, since later we only want to make one single directory available to our network clients. The apt-mirror tool however creates one directory per official mirror, which makes it more complex and is not shown here. So avoid the use of several different mirror servers and only use a single one if possible.
  3. Make sure your have PLENTY of disk space available for each distro release and repositories you select. Depending on your choices it can easily exceed 500GB total as in my example here.

Installation and configuration:

  1. Install apt-mirror:
    sudo apt install -y apt-mirror
  2. Prepare your download/mirror directory:
    mkdir -p /pentyOfSpaceHere/apt-mirror
    chown -c apt-mirror.users /pentyOfSpaceHere/apt-mirror
    chmod -c u+rwx /pentyOfSpaceHere/apt-mirror
  3. Make your own version of /etc/apt/mirror.list and put the gathered information in:
    cp /etc/apt/mirror.list /etc/apt/mirror.list.$(date +%F)
    
    cat << 'EOF' > /etc/apt/mirror.list
    ############# config ##################
    #
    # set base_path    /var/spool/apt-mirror
    set base_path    /pentyOfSpaceHere/apt-mirror
    #
    # set mirror_path  $base_path/mirror
    # set skel_path    $base_path/skel
    # set var_path     $base_path/var
    # set cleanscript $var_path/clean.sh
    # set defaultarch  <running host architecture>
    # set postmirror_script $var_path/postmirror.sh
    # set run_postmirror 0
    set nthreads     20
    set _tilde 0
    #
    ############# end config ##############
    
    ##################################
    ##
    ##  UBUNTU 16.04 LTS (xenial)
    ##
    ##################################
    
    deb http://de.archive.ubuntu.com/ubuntu/ xenial main restricted universe multiverse 
    #deb-src http://de.archive.ubuntu.com/ubuntu/ xenial main restricted universe multiverse 
    deb http://de.archive.ubuntu.com/ubuntu/ xenial-updates main restricted universe multiverse 
    deb http://de.archive.ubuntu.com/ubuntu/ xenial-backports main restricted universe multiverse 
    deb http://de.archive.ubuntu.com/ubuntu/ xenial-security main restricted universe multiverse 
    deb http://archive.canonical.com/ubuntu/ xenial partner
    
    ##################################
    ##
    ##  UBUNTU 18.04 LTS (bionic)
    ##
    ##################################
    
    deb http://de.archive.ubuntu.com/ubuntu/ bionic main restricted universe multiverse 
    #deb-src http://de.archive.ubuntu.com/ubuntu/ bionic main restricted universe multiverse 
    deb http://de.archive.ubuntu.com/ubuntu/ bionic-updates main restricted universe multiverse 
    deb http://de.archive.ubuntu.com/ubuntu/ bionic-backports main restricted universe multiverse 
    deb http://de.archive.ubuntu.com/ubuntu bionic-security main restricted universe multiverse 
    deb http://archive.canonical.com/ubuntu bionic partner
    
    
    ##################################
    ##
    ##  UBUNTU 19.04 (disco)
    ##
    ##################################
    
    deb http://de.archive.ubuntu.com/ubuntu/ disco main restricted universe multiverse 
    #deb-src http://de.archive.ubuntu.com/ubuntu/ disco main restricted universe multiverse 
    deb http://de.archive.ubuntu.com/ubuntu/ disco-updates main restricted universe multiverse 
    deb http://de.archive.ubuntu.com/ubuntu/ disco-backports main restricted universe multiverse 
    deb http://de.archive.ubuntu.com/ubuntu disco-security main restricted universe multiverse 
    deb http://archive.canonical.com/ubuntu disco partner
    
    ##################################
    ##
    ##  GLOBAL / CLEANUP
    ##
    ##################################
    
    clean http://de.archive.ubuntu.com/ubuntu
    clean http://archive.canonical.com/ubuntu
    
    EOF
  4. Apt-mirror does not come with a postmirror.sh script, even though its called by default on every mirroring. So to prevent the error message, we create our own script:
    cat << EOF > /pentyOfSpaceHere/apt-mirror/var/postmirror.sh
    #!/bin/bash -e
    
    ## Anything in this file gets run AFTER the mirror has been run.
    ## Put your custom post mirror operations in here (like rsyncing the installer
    ## files and running clean.sh automatically)!
    
    EOF
  5. For automatic updating your local mirrors, use crontab. In my special case, i only want to update my local mirrors once a month:
    cat << EOF > /etc/cron.d/apt-mirror
    #
    # Regular cron jobs for the apt-mirror package
    #
    #m      h       dom     mon     dow     user            command
    0       1       1       *       *       apt-mirror      /usr/bin/apt-mirror > /var/log/apt-mirror-cron.log
    
    EOF
  6. :!: WARNING: The first run of apt-mirror will download several hundred GB of files and will load down your internet connection for hours or days! Make sure this is no problem for you.
  7. Start the first run/download from the official mirrors with:
    # this will run apt-mirror as service user 'apt-mirror'
    # use CTRL+C to abort download if needed. Else let it do its job and 
    # check back 24h later.
    #
    su - -c /usr/bin/apt-mirror apt-mirror
  8. If the download is done, your local repo has been created and should be ready at /pentyOfSpaceHere/apt-mirror/mirror/yourOfficialMirrorChosen/ubuntu/ on your servers filesystem.
  9. To access it from your local network by clients you'll need some sort of webserver or ftp server. Its your choice. Just make sure, it can access (read only) your mirrored files using its service account. This manual assumes we use Apache and use its Ubuntu default webserver directory:
    # make symbolic link for apache to reach the mirror dir.
    ln -s /pentyOfSpaceHere/apt-mirror/mirror/yourOfficialMirrorChosen/ubuntu /var/www/ubuntu
  10. In my example my Ubuntu mirror should be available to my network clients under http://myServer/ubuntu/ and can now be used on Ubuntu (or debian) Hosts in their /etc/apt/sources.list like this:
    deb http://myServer/ubuntu/ xenial main restricted
    deb http://myServer/ubuntu/ xenial-updates main restricted
    deb http://myServer/ubuntu/ xenial universe
    deb http://myServer/ubuntu/ xenial-updates universe
    deb http://myServer/ubuntu/ xenial multiverse
    deb http://myServer/ubuntu/ xenial-updates multiverse
    deb http://myServer/ubuntu/ xenial-backports main restricted universe multiverse

I hope this helps someone.

Axel Werner 2019-10-05 18:26

it-artikel/linux/how-to-make-an-ubuntu-or-debian-apt-repository-locally-available-as-an-emergency-offline-fallback.txt · Last modified: 2019-10-10 18:05 by mail@awerner.myhome-server.de