LinuxQuimper

Ceci est une ancienne révision du document !

---

Nous utiliserons squid qui permet de réecrire certaines url (celles correspondant aux dépots) grace à un script python.

Exemple :
- on demande http://archive.ubuntu.com/ubuntu/pool/a/alien_1.2.3.deb
- Squid transmet l'url au script python
- le script retranscris en adresse locale : http://127.0.0.1/ubuntu/pool/a/alien_1.2.3.deb
- Si le fichier local existe, le script renvoi l'url locale, sinon renvoi l'url non modifiée

Voici le script en question /usr/local/bin/squid-redirector.py :

#!/usr/bin/env python

#    This program is free software: you can redistribute it and/or modify
#    it under the terms of the GNU General Public License as published by
#    the Free Software Foundation, either version 3 of the License, or
#    (at your option) any later version.
#
#    This program is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#    GNU General Public License for more details.
#
#    You should have received a copy of the GNU General Public License
#    along with this program.  If not, see <http://www.gnu.org/licenses/>.
#               
#                  
#    Author : Fanch317 (fanch@linuxquimper.org)
#
#    Date : 2009-09-27
#    Updated : 2009-09-28
#
#    Description : Ce script est un redirecteur pour squid.
#                  Il est destinne a faire passer toute requete de depots de 
#                  distributions vers des depots locaux plus rapides lors 
#                  d'install-party. (Proxy transparent)

import sys,re,os.path,urllib2,logging

redirectors = {}
redirectors[0] = ["(.*\.)?archive.ubuntu.com/ubuntu", "http://127.0.0.1/ubuntu/"]
#redirectors[i+1] = ["Partern de l'url a catcher", "URL locale des fichiers"]


LOG_FILENAME = '/var/log/squid/squid-redirector.log'
logging.basicConfig(filename=LOG_FILENAME,level=logging.DEBUG)

logging.debug('Lancement d\'thread squid-redirector.py')


def process_url(line_squid):
  try:
    list = line_squid.split(' ')
    url_input = list[0]
#    logging.debug('URL - '+list[0] + ' -')
    try:
      if re.search('http://' + redirectors[0][0],url_input):
        logging.debug('L\'url correspond a un depot')
        try:
          url_locale = re.sub(r'http://' + redirectors[0][0],redirectors[0][1],url_input)
          logging.debug('On verifie la presence du fichier local : '+url_locale)
          urllib2.urlopen(url_locale)
          url_output = url_locale + '\n'
          logging.debug('Present')
        except:
          url_output = url_input + '\n'
          logging.debug('Absent')
      return url_output
    except:
      logging.debug('Aucun depot de correspond')
      return url_input + '\n'
  except:
    logging.debug('Impossible de recuperer l\'url')
    return '\n' 

while True:
  # URL ip-address/fqdn ident method
  line_squid = sys.stdin.readline().strip()
  logging.debug('Reception d\'une requete : '+line_squid)
  url_processing = process_url(line_squid)
  sys.stdout.write(url_processing)
  sys.stdout.flush()

Modifiez la config de squid de la façon suivante :

url_rewrite_program /usr/local/bin/squid-redirector.py

Le mirroir ubuntu peut se télécharger grace à debmirror avec la configuration suviante :

#!/bin/bash

## Setting variables with explanations.

#
# Don't touch the user's keyring, have our own instead
#
export GNUPGHOME=/home/mirrorkeyring

# Arch=         -a      # Architecture. For Ubuntu can be i386, powerpc or amd64.
# sparc, only starts in dapper, it is only the later models of sparc.
#
arch=i386,amd64

# Minimum Ubuntu system requires main, restricted
# Section=      -s      # Section (One of the following - main/restricted/universe/multiverse).
# You can add extra file with $Section/debian-installer. ex: main/debian-installer,universe/debian-installer,multiverse/debian-installer,restricted/debian-installer
#
section=main,restricted,universe,multiverse,main/debian-installer,universe/debian-installer,multiverse/debian-installer,restricted/debian-installer

# Release=      -d      # Release of the system (Dapper, Edgy, Feisty, Gutsy, Hardy, Intrepid), and the -updates and -security ( -backports can be added if desired)
#
release=jaunty,karmic

# Server=       -h      # Server name, minus the protocol and the path at the end
# CHANGE "*" to equal the mirror you want to create your mirror from. au. in Australia  ca. in Canada.
# This can be found in your own /etc/apt/sources.list file, assuming you have Ubuntu installed.
#
server=archive.ubuntu.com

# Dir=          -r      # Path from the main server, so http://my.web.server/$dir, Server dependant
#
inPath=/ubuntu

# Proto=        -e      # Protocol to use for transfer (http, ftp, hftp, rsync)
# Choose one - http is most usual the service, and the service must be avaialbe on the server you point at.
#
proto=http

# Outpath=              # Directory to store the mirror in
# Make this a full path to where you want to mirror the material.
#
outPath=/mnt/datas/repositories/ubuntu/archive/

# The --nosource option only downloads debs and not deb-src's
# The --progress option shows files as they are downloaded
# --source \ in the place of --no-source \ if you want sources also.
# --nocleanup  Do not clean up the local mirror after mirroring is complete. Use this option to keep older repository
# Start script
#


CMD="debmirror -a $arch --no-source -s $section -h $server -d $release -r $inPath -v --progress --postcleanup -e $proto --ignore-release-gpg $outPath"

echo "Lancement de la commande : "
echo $CMD

${CMD}


#### End script to automate building of Ubuntu mirror ####