Axel Werner's OPEN SOURCE Knowledge Base

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== How to make an Ubuntu or Debian apt repository locally available as an emergency offline fallback ======


Maybe you never had that problem yourself, but there may be reasons why someone want to have one or multiple Ubuntu or Debian online repositories available on your local network. Maybe because you throw an Linux installation party and dozens of times the same repos and files are been pulled over a thin DSL connection. Or you want your own local mirror in case your internet connection goes down. Or you might just want to keep a certain version or snapshot of the online repo for a specific UBUNTU release or distro, because one day it may become unavailable, if UBUNTU/DEBIAN decides to take it down if its "too old" for them. This might be interesting for some "retro computing" guys as well.

So how to achieve that ?

As always, there is more than just one solution. However... this is how i did it.

===== Pre-Requirements: =====

  - Define what distro releases and what "repositories" or "sections" you want to have available on your LAN. With "repositories" or "sections" i mean those "keywords" that follow one of your "deb" lines in your **/etc/apt/sources.list**. In my example i want to have Ubuntu 16.04 LTS, Ubuntu 18.04 LTS and Ubuntu 19.04 available at all time. So ill need the code names for its respositories/sections later. The format is as shown here: <code>
#deb url distroReleaseName repo/sections
deb http://archive.ubuntu.com/ubuntu/ disco main restricted universe multiverse
</code>
  - Take a note what official Ubuntu/Debian mirror server your want to use for your download. For me it will be **%%http://de.archive.ubuntu.com/ubuntu/%%**
    * :!: **BEWARE: This manual assumes the use of only ONE official mirror server, for all distros or releases we want! This is somewhat important, since later we only want to make one single directory available to our network clients.** The **apt-mirror** tool however creates one directory per official mirror, which makes it more complex and is not shown here. So avoid the use of several different mirror servers and only use a single one if possible.
  - Make sure your have PLENTY of disk space available for each distro release and repositories you select. Depending on your choices it can easily exceed 500GB total as in my example here.

===== Installation and configuration: =====

  - :!::!::!: **BUG: It seems that the ''apt-mirror'' package that comes with UBUNTU repositories up to recently 1/2020 contain a major BUG, that does'nt do a proper mirroring job! This results in local mirrors that will cause 404 errors about not finding "cnf" and other files when doing a ''apt update'' on your client side. In addition, for some reason the original ''apt-mirror'' package/source at the moment seems to be orphaned and people are about to fork and reorganize this open source project. So we are NOT USING THE UBUNTU VERSION OF ''APT-MIRROR'', Instead we use some fork on GitHub. So the risk of installing software from unknown or untrusted sources is on you!**  --- //[[mail@awerner.myhome-server.de|Axel Werner]] 2020-01-24 15:08//
  - Install the special **apt-mirror** fork from [[https://github.com/josbraden/apt-mirror]] by: <code>
cd /opt/
git clone https://github.com/josbraden/apt-mirror
ln -vs /opt/apt-mirror/apt-mirror /usr/bin/apt-mirror
</code> 
  - Prepare your download/mirror directory: <code>
mkdir -p /pentyOfSpaceHere/apt-mirror
chown -c apt-mirror.users /pentyOfSpaceHere/apt-mirror
chmod -c u+rwx,g+rwxs /pentyOfSpaceHere/apt-mirror
</code>
  - Make your own version of **/etc/apt/mirror.list** and put the gathered information in:<code>
cp /etc/apt/mirror.list /etc/apt/mirror.list.$(date +%F)

cat << 'EOF' > /etc/apt/mirror.list
############# config ##################
#
set base_path    /pentyOfSpaceHere/apt-mirror
#
# set mirror_path  $base_path/mirror
# set skel_path    $base_path/skel/etc/apt/sources.list.d/localRepo-ubuntu-2004-focal.list
# set var_path     $base_path/var
set cleanscript $var_path/clean.sh
# set defaultarch  <running host architecture>
# set postmirror_script $var_path/postmirror.sh
# set run_postmirror 0
set nthreads     10
set _tilde 0
#
############# end config ##############

##################################
##
##  UBUNTU 16.04 LTS (xenial)
##
##################################

deb http://archive.ubuntu.com/ubuntu xenial main restricted universe multiverse 
deb http://archive.ubuntu.com/ubuntu xenial-updates main restricted universe multiverse 
deb http://archive.ubuntu.com/ubuntu xenial-backports main restricted universe multiverse 
deb http://archive.ubuntu.com/ubuntu xenial-security main restricted universe multiverse 
deb http://archive.canonical.com/ubuntu xenial partner

deb-i386 http://archive.ubuntu.com/ubuntu xenial main restricted universe multiverse 
deb-i386 http://archive.ubuntu.com/ubuntu xenial-updates main restricted universe multiverse 
deb-i386 http://archive.ubuntu.com/ubuntu xenial-backports main restricted universe multiverse 
deb-i386 http://archive.ubuntu.com/ubuntu xenial-security main restricted universe multiverse 
deb-i386 http://archive.canonical.com/ubuntu xenial partner

##################################
##
##  UBUNTU 18.04 LTS (bionic)
##
##################################

deb http://archive.ubuntu.com/ubuntu bionic main restricted universe multiverse 
deb http://archive.ubuntu.com/ubuntu bionic-updates main restricted universe multiverse 
deb http://archive.ubuntu.com/ubuntu bionic-backports main restricted universe multiverse 
deb http://archive.ubuntu.com/ubuntu bionic-security main restricted universe multiverse 
deb http://archive.canonical.com/ubuntu bionic partner

deb-i386 http://archive.ubuntu.com/ubuntu bionic main restricted universe multiverse 
deb-i386 http://archive.ubuntu.com/ubuntu bionic-updates main restricted universe multiverse 
deb-i386 http://archive.ubuntu.com/ubuntu bionic-backports main restricted universe multiverse 
deb-i386 http://archive.ubuntu.com/ubuntu bionic-security main restricted universe multiverse 
deb-i386 http://archive.canonical.com/ubuntu bionic partner


##################################
##
##  UBUNTU 20.04 LTS (focal)
##
##################################

deb http://archive.ubuntu.com/ubuntu focal main restricted universe multiverse 
deb http://archive.ubuntu.com/ubuntu focal-updates main restricted universe multiverse 
deb http://archive.ubuntu.com/ubuntu focal-backports main restricted universe multiverse 
deb http://archive.ubuntu.com/ubuntu focal-security main restricted universe multiverse 
deb http://archive.canonical.com/ubuntu focal partner

deb-i386 http://archive.ubuntu.com/ubuntu focal main restricted universe multiverse 
deb-i386 http://archive.ubuntu.com/ubuntu focal-updates main restricted universe multiverse 
deb-i386 http://archive.ubuntu.com/ubuntu focal-backports main restricted universe multiverse 
deb-i386 http://archive.ubuntu.com/ubuntu focal-security main restricted universe multiverse 
deb-i386 http://archive.canonical.com/ubuntu focal partner




##################################
##
##  GLOBAL / CLEANUP
##
##################################

clean http://archive.ubuntu.com/ubuntu
clean http://archive.canonical.com/ubuntu




EOF
</code>
  - Apt-mirror does not come with a **postmirror.sh** script, even though its called by default on every mirroring. So to prevent the error message, we create our own script:<code>
cat << EOF > /pentyOfSpaceHere/apt-mirror/var/postmirror.sh
#!/bin/bash -e

## Anything in this file gets run AFTER the mirror has been run.
## Put your custom post mirror operations in here (like rsyncing the installer
## files and running clean.sh automatically)!

EOF
</code>
  - For automatic updating your local mirrors, use crontab. In my case, i want to update my local mirrors once a day at 1am: <code>
cat << EOF > /etc/cron.d/apt-mirror
#
# Regular cron jobs for the apt-mirror package
#
#m	h	dom	mon	dow	user		command
0	1	*	*	*	apt-mirror	/usr/bin/apt-mirror 2>&1 | tee -a /var/log/apt-mirror-cron.log

EOF
</code>
  - :!: **WARNING:** The first run of **apt-mirror** will download several hundred GB of files and will load down your internet connection for hours or days! Make sure this is no problem for you.
  - Start the first run/download from the official mirrors with:<code>
# this will run apt-mirror as service user 'apt-mirror'
# use CTRL+C to abort download if needed. Else let it do its job and 
# check back 24h later.
#
su - -c /usr/bin/apt-mirror apt-mirror
</code>
  - If the download is done, your local repo has been created and should be ready at **/pentyOfSpaceHere/apt-mirror/mirror/yourOfficialMirrorChosen/ubuntu/** on your servers filesystem. 
  - To access it from your local network by clients you'll need some sort of webserver or ftp server. Its your choice. Just make sure, it can access (read only) your mirrored files using its service account. This manual assumes we use Apache and use its Ubuntu default webserver directory: <code>
# make symbolic link for Apache to reach the mirror dirs.
ln -s /pentyOfSpaceHere/apt-mirror/mirror/yourOfficialMirrorChosen/ubuntu /var/www/ubuntu
ln -s /pentyOfSpaceHere/apt-mirror/mirror/archive.canonical.com/ubuntu /var/www/ubuntuPartner
</code>
  - In my example my Ubuntu mirror should be available to my network clients under **%%http://myServer/ubuntu/%%** and can now be used on Ubuntu (or debian) Hosts in their **/etc/apt/sources.list** like this:<code>

# Example for UBUNTU 19.04 DISCO
deb http://myLocalServer/ubuntu/ disco main restricted universe multiverse
deb http://myLocalServer/ubuntu/ disco-updates main restricted universe multiverse
deb http://myLocalServer/ubuntu/ disco-security main restricted universe multiverse
deb http://myLocalServer/ubuntu/ disco-backports main restricted universe multiverse

deb http://myLocalServer/ubuntuPartner/ disco partner


</code> OR you can just drop an additional file into **/etc/apt/sources.list.d/** with the additional apt repositories you want to use: <code>
cat << EOF > /etc/apt/sources.list.d/localRepo-ubuntu-2004-focal.list

# Example for UBUNTU 20.04 FOCAL
deb http://myLocalServer/ubuntu/ focal main restricted universe multiverse
deb http://myLocalServer/ubuntu/ focal-updates main restricted universe multiverse
deb http://myLocalServer/ubuntu/ focal-security main restricted universe multiverse
deb http://myLocalServer/ubuntu/ focal-backports main restricted universe multiverse

deb http://myLocalServer/ubuntuPartner/ focal partner

EOF
</code>
  - Now do a ''sudo apt update'' and it should not throw any errors at you, if you have your Apache webserver configured well and it can access the mirrored files via the **URL defined in the sources.list** file.

I hope this helps someone.

 --- //[[axel.werner.1973@gmail.com|Axel Werner]] 2020-10-08 13:34//

{{tag>linux ubuntu debian apt aptitude mirror repo repository offline apt-mirror}}