How To Proxy The Apt-Get Package Manager

How To Proxy The Apt-Get Package Manager

Get Social!

If you’re using Debian, Ubuntu, or any similar distribution which uses apt-get as it’s package manager and you’re behind a http proxy then you’re going to need to tell apt-get what your proxy settings are.

The way you tell apt-get what your proxy settings are is simple; set an environment variable and apt-get will pick it up and do the rest.

Set an environment variable called http_proxy and specify your http proxy server protocol (http or https), ip address and port. apt-get will then direct all of it’s fetch operations through the http proxy using the specified details. Use the following syntax to export the http_proxy variable and substitute IP and PORT for your proxy details.

export http_proxy=http://IP:PORT

For example, if your proxy uses the ip 10.10.10.10 and port 8080 then you’d write the following:

export http_proxy=http://10.10.10.10:8080

If your proxy requires user authentication then use the below syntax, substituting USER and PASSWORD for your proxy authentication details.

export http_proxy=http://USER:[email protected]:8080

You can also add this to your .bash_profile so that the variable is set each time you log in. Just remember to update your password if it ever changes!

vi ~/.bash_profile

Proxy the Proxmox Web GUI with Nginx Over HTTPS with Load Balancing

Get Social!

The Proxmox web GUI is served by Proxmox’s new event driven API server called PVE Proxy. The default settings for the Proxmox web GUI is to listen on port 8006 for incoming HTTPS connections.

The following tutorial will show you how to use Nginx to reverse proxy the PVE Proxy application to do the following:

  • Redirect HTTP requests to use the HTTPS protocol.
  • Add your own certificate to use for HTTPS.
  • Listen on the standard HTTPS port (port 443).

The following steps show how to use Nginx to reverse Proxy Proxmox’s web GUI. If you would prefer to use Apache, please see my other blog post.

The first step is to make sure you have Nginx installed on the machine, or virtual instance, that you are going to use. You can install Nginx directly on the Proxmox host however, I prefer to keep the host software as standard as possible and run all additional applications in OpenVZ containers.

Create a shell session on the machine you would like to use and use apt-get to install Nginx.

apt-get install nginx

Make sure you have an SSL certificate and key pair on your server. See my OpenSSL certificate cheat sheet for more information.

We now need to specify the configuration for Nginx. Remove the existing site configuration and create a new configuration file called proxmox-gui. You can call this file whatever you wish, but you will also need to use the same name in the below steps.

rm -f /etc/nginx/sites-enabled/default
vi /etc/nginx/sites-enabled/proxmox-gui

Add the below text to your proxmox-gui file. You will need to substitute some of the settings with your own values:

  • ssl_certificate – this should point to your SSL certificate to use for signing the SSL traffic.
  • ssl_certificate_key – is this key which matches the above certificate.
  • server – this is the IP and port of your Proxmox server. If you have installed Nginx on the same host as the Proxmox web GUI then you could use https://localhost:8006 here.
upstream proxmox {
    server 10.10.10.10:8006;
}

server {
    listen 80 default_server;
    rewrite ^(.*) https://$host$1 permanent;
}

server {
    listen 443;
    server_name _;
    ssl on;
    ssl_certificate /etc/nginx/ssl/cert.pem;
    ssl_certificate_key /etc/nginx/ssl/key.pem;
    proxy_redirect off;
    location / {
        proxy_pass https://proxmox;
    }
}

If you have multiple Proxmox servers in a cluster, it would make sense to use load balancing in Nginx. We don’t really want to use this feature to spread the load, because usually the traffic will be very light – we want to use it so that if one node in the cluster is down, Nginx will automatically try a different node in the cluster.

To add load balancing, add your additional servers in the upstream proxmox code section. For example:

upstream proxmox {
    server 10.10.10.10:8006;
    server 10.10.10.11:8006;
    server 10.10.10.12:8006;

}

We need to link the newly created config file so that Nginx can load it.

ln -sf /etc/nginx/sites-available/proxmox-gui /etc/nginx/sites-enabled/

The last step is to restart Nginx web server to pick up the new settings.

service nginx restart

Your Proxmox web GUI should now be available on the IP address of your Nginx server on the HTTPS protocol.

 


Speed up Multiple apt-get install Requests by Caching the Repository

Get Social!

Linux penguinapt-get is the tool used in Debian and Ubuntu to manage packages installed on the system. Each time an update is available, or you install a new package the files will be downloaded from one of the central repository servers out on the internet and installed on your system.

There are two main problems with this:

  • Your servers may not be on able to access the internet directly for security reasons
  • Installing the same package on multiple servers will result in downloading the package the same amount of times. This could be slow or expensive in terms of bandwidth.

To solve the problem you can mirror the source repository on your own local server and add that as a source for your servers to update from. The main issue with this is that each distribution has a huge catalog of package which would take up vast amounts of space. Multiply this by the different releases of operating system in your environment and you could be talking terabytes of space.

Various utilities have been created to work round this problem such as apt-proxy, apt-cacher and debproxy. These utilities work by only caching some of the most used packages and fetching the rest from the source.

The below example will use apt-cacher-ng which is a middle man who sits in between the server being updated and the source repository out on the internet. It chooses to cache some regularly or recently used files locally and will recall them when they are requested which greatly speeds up the process for the requesting machine. The cache is frequently cleared to make sure that disk space is only being used for the most necessary packages. This drastically reduces resource required to run the service, whilst speeding up package downloading and guaranteeing that all packages are available.

Setting up apt-cacher-ng server

The apt-cacher-ng utility sits on a server which must be able to access both the public network and any internal network which your other servers may sit on.

Run apt-get install to install the proxy utility.

apt-get install apt-cacher-ng

The default installation of apt-cacher-ng holds details of both Ubuntu and Debian source repositories and is ready to use.

If you need to change the settings of the application such as the port it listens on, edit the below file:

vi /etc/apt-cacher-ng/acng.conf

You can now access the web interface using using the local machine’s IP or host name and the port. The default port is 3142.

apt-cache-nr-homescreen

This page shows that apt-cacher-ng is working correctly and is ready to cache the first source requests.

The next step is to add the server location to your clients. Create the below file and add details of your caching server.

/etc/apt/apt.conf.d/02proxy

Add the below line and edit [SERVER_IP] and [SERVER_PORT] to match your apt-cacher-ng configuration.

Acquire::http { proxy "http://[SERVER_IP]:[SERVER_PORT]"; };

Eg.

Acquire::http { proxy "http://10.10.10.1:3142"; };

Finally, run the update command on your clients to cause the proxy to cache the package lists. Packages will also be cached soon as you start to install or updates packages on your client.

To make sure that apt-cacher-ng is doing it’s job, tail the log to make sure entries are appearing.

tail -f /var/log/apt-cacher-ng/apt-cacher.log

In addition, you can also view the webpage for statistics on cache hits and misses:

http://[SERVER_IP]:[SERVER_PORT]/acng-report.html?doCount=Count+Data#top


Using a Proxy Server with Java

Tags :

Category : How-to

Get Social!

java-logoJava applications can use a proxy server for making HTTP/ HTTPS connections to the internet by adding additional arguments to the startup command. It’s the JVM which is created with additional arguments to set up the JVM with the details required to proxy requests.

The following arguments are required:

  • http.proxyHost – the host or IP address of the proxy server.
  • ptty.proxyPort – is the port used by the proxy server. If this is not used the default of port 80 is assumed.

For example, to start the application myApplication.jar with a proxy server located at myproxyserver.local on port 8080:

java -jar myApplication.jar -Dhttp.proxyHost=myproxyserver.local  -Dhttp.proxyPort=8080

You can also specify the nonProxyHosts to exclude specific domains from using the proxy server. The below will not proxy anything on mydomain.com, server.local or localhost.

-Dhttp.nonProxyHosts="*.mydomain.com|server.local|localhost"

Visit our advertisers

Quick Poll

Are you using Docker.io?

Visit our advertisers