1. December 2018

Nextcloud on a Raspberry Pi

There is a lot on the 'net about running your own sync and other cloud services with Nextcloud and the Raspberry Pi, many concluding it's not viable for continuous use, or many losing interest after a while. In fact, with some thought-through choices, a production environment for, say 5 or fewer users is perfectly feasible.


This blog has a number of entries regarding the viability of the Raspberry Pi as a general purpose Linux server for a small network. Our main driver for exploring the Pi as such an option is the fact that we are off-grid, and must therefore be careful about power consumption, especially for a device that will be left on 24 hours a day.  We have now run a Pi as a server for our own network services for nearly 3 years. You can read about the migration and what it did for us here.

Meanwhile, from those days, the Raspberry Pi foundation continues to churn out little tweaks that constantly improve the performance of the Pi, and the latest version, the Raspberry Pi 3 Model B+, has a number of little tweaks that make life a little easier.  Among these are a slightly faster processor and, alongside the slightly slower Model 3B, the ability to do away with the microSD card to boot the device.  I had commented in a previous article the necessity of being a somewhat open-minded system administrator and not dismissing the idea of relying on USB for the disk, which would indeed be best practice, but if you think about it, all disks are plugged on somewhere along the line, so the additional risk is not necessarily a huge one.

There are now options of being able to download disk images of Nextcloud running on optimised setup for the Pi. These options may suit many people.  However, one is then dependant on the future of those projects, and perhaps some options you would like to do differently are not available or too difficult. In the ethos of Linux, the Raspberry Pi and Nextcloud, I prefer to be in control of my own decisions when installing such services.

Another recent improvement, which transforms the performance of Nextcloud on the Pi, is PHP version 7.  But there are still absolute limits, for example,  977MB of available memory, that need to be kept in mind.  So what optimisations can be done in providing a LAMP (Linux Apache Mysql (Mariadb) PHP) stack to make it reliable and usable?

I know enough about Debian, or its Raspberry Pi cousin, Raspbian, to know how to install or strip down to a bare minimum, and ensure there are no extraneous services I do not want on the server.  The deborphan package is helpful to this.   As you remove things that are not necessary, like the triggerhappy daemon, other things that are not really needed on a server, left-over -dev files and so on, you can run "deborphan --guess-all" to give you an idea of dangling packages. Using this, you can get a basic, but still standard version of Raspbian Stretch, the current version, down to around 30MB of memory or less.

I had looked at the possibility of using Lighttpd or Nginx as the web server, but when running a PH application like Nextcloud, the web server potentially has little to do with performance. In addition, Apache's support for local security and other localisations through the use of .htaccess files does make life easier. I use the mpm_event option, and alter the number of available apache processes to a low number suitable for supporting a low number of users.

MySQL or Mariadb is more of a challenge. The old MySQL 5.7 that came with Raspbian Jessie could be stripped down to use a reasonable amount of memory, but mariadb just needs more.  MySQL tends to assume that the system it is running on is only used for that purpose, and all the tuning sites and pages ask for details of the total memory the system has.  However, as a long time techie, I have to admit that, when I have the option, I prefer using something other than MySQL.  And before Nextcloud came along, it's parent, Owncloud, used to throw horrible wobblies with MySQL whenever the system was upgraded.  I discovered that using Postgresql as an alternative, which is fully supported, though Nextcloud  recommends MySQL or MariaDB, and many of the update problems disappeared. I also had the feeling, which is hard to prove, that Postresql performed slightly better.  But, of greater value on a Pi, the memory use of Postgresql is much more manageable than MySQL, even with drastic memory optimisations. What is more, memory use does not seem to fluctuate as much as Mariadb, again, making running on a Pi easier.

There is one additional optimisation that is difficult to prove, but is a result of years of experience.  I wrote a while back about Linux filesystems, noting in particular that the venerable JFS is the only filesystem with which I have never lost data. in addition, an an out-of-date  benchmark comparing filesystems noted that not only did JFS get most space from a given partition or disk, but that it was gentlest on the processor.  These are important little wheezes when running a Pi. Unfortunately,  Raspbian and the Pi's firmware do not support JFS out of the box.  It is necessary to install an initrd file, as though the system were an x86 system.  This used to be a real pain because the FAT filesystem where the boot takes place does not support file links, so as the initrd changes name when the kernel changes, a lot of manual work has to follow. Fortunately, someone has resolved the problem.  This link has a file which, when dropped into /etc/kernel/postinst.d/ creates a new initramfs.gz file automatically.  This makes running JFS for data security entirely possible and reliable, BUT there is an additional advantage.  A number of benchmarks over the years have shown that the Postgresql database really enjoys the JFS filesystem, so it is a way of getting a little more performance from Postgresql as well.

Nextcloud calls for a local PHP cache, and one option is redis.  As most techies will know, Redis has been altering its licence terms as a way of getting more money from large scale users of the product. History shows that when companies that had until that point understood what free and open source software was about made such changes, the writing is on the wall, and it is only a matter of time until that company or software falls into obscurity.  Anyway, on a small system, it is doubtful that redis makes that much difference in comparison to simpler options, so PHP is enable with opcache and apcu.  Nextcloud helpfully suggests settings for these.

I run php-fpm rather than the apache PHP module. This has long been known to provide better performance, but it can also be set to spin up on demand rather than having to allocate chunks of resource to php.

The result is a system which, with a few users, after a week of activity, database backups etc, uses around 100MB of the 977MB available.  The only time memory use is of concern is if someone drops a very large image into the gallery app, and imagemagick has to crunch for a while to resize the image.

There is, however, one aspect that can make the system look laggy, and that is when logging in when the brute-force settings plugin is installed.  Whatever that useful security option does to prevent bad actors from trying to access your data, it crunches  away for a few seconds before logging the user in. This is only from the web interface; the sync clients on desktops or phones/tablets are unaffected and work fast.  After logging in, moving from application to application, while not instantaneous or even lightning fast, is perfectly acceptable and not likely to slow one down.

Apart from our own instance of Nextcloud, I have installed a system based on the above principles elsewhere, and between the two systems, around 3TB of data is accessible.  I completed the second installation in Edinburgh over the weekend, and the sense of achievement when the security check returned a green A+, combined with the users commenting on the additional speed and usability, was pretty gratifying.

I think this experience is enough to conclude that, while of course a Raspberry Pi would be slower than a multi-core Xeon server with shedloads of memory, it is perfectly adequate for use for small workgroups, but only if following a path slightly less travelled than the defaults. None of the above optimisations is hard or whacky. 

I hope these ideas are helpful.

16. October 2018

Revisiting server-based antispam with Bogofilter

Some simple maintenance makes life interesting, but simplicity saves the day

Continue reading

2. October 2018

Latest update - off-grid Linux IT services

The latest (October 2018) update on hardware and software choices when your power supply is limited

This is a technical blog post which may not be of general interest, and assumes a certain level of technical understanding

Continue reading

18. August 2018

Off-grid - making do and mending

A bit of resourcefulness and "it'll-come-in-handy-some-day" hoarding saves the day

Continue reading

2. May 2018

Weather Station Software

Our weather data goes back to 2011. How to maintain that collection while changing the weather station software

Continue reading

5. February 2018

Raspberry Pi home server - setting up OpenDKIM with postfix

Some vague notes about setting up openDKIM against multiple domains on a single instance, all domains using one key.

Continue reading

14. September 2017

Preparing digital photo files for use on the Web

Reducing size, adding copyright information and other watermarking is useful for pictures one may wish to use on the Internet. Selecting the images, or using drag 'n' drop graphically is easier than pushing specific files through a script. 

Note - I am no programmer, and my scripting abilities are severely limited. This works for me, but may not work for you. Use these ideas at your own risk.

-----

Continue reading

23. August 2017

Lightweight anti-spam alternative for small servers

Spamassassin may be the standard anti-spam utility for servers, but it can't be considered to be either fast or low on resources.  Bogofilter may offer some advantages, but unlike spamassassin, tutorials and how-to's are thin on the ground

Warning: this is a technical post, full of jargon and an expectation that it will be read by system administrators, so may not be of interest to all readers.

Integrating postfix, dovecot and bogofilter on a Raspberry Pi.

Edit: Some months have passed since installing bogofilter.  It is not as fast as a daemon, but not as slow as spamassassin, either.  As expected, it has taken a couple of months to build up accuracy, but the system is excellent, and now reliably marks some spam that always got through spamassassin. This seems a good way forward for lightweight email systems.

Continue reading

16. July 2017

The Calendar Hokey Cokey

A problem with the Nextcloud calendar can be resolved with a lot of to-ing and fro-ing, thanks to standards

Continue reading

16. June 2017

A Digital Re-think

Some thoughts on images in the digital world...

But note, as with all things photographic, this is all just opinion.

Continue reading

30. March 2017

A little dab in the wrong plaice makes you wonder...

Flying fish in the hills of Assynt

Continue reading

The Queen's Mite

Finding and photographing a bumble bee loaded with mites

Continue reading

26. February 2017

Lexie 2003 - 2017

TinSlave-154702-23022017.jpg

The life and times of a wonderful Westie


Continue reading

27. January 2017

Tales of experience

Some stories from more than 25 years in corporate IT, some of which seem quite historical now

Continue reading

1. October 2016

Serving up some Pi

You really can run on a Raspberry Pi

Continue reading

13. July 2016

A new battery bank for our off-grid power supply

TinSlave-113241-13072016.jpg

The final piece of the recent power revamp

Continue reading

9. April 2016

The Silver Darling - When you need a good Linux laptop

TinSlave-104713-09042016.jpg

The Entroware Apollo laptop is a good option to be sure that new hardware runs well with Linux, and the system itself is good. But the process of buying a Linux laptop could be more straight forward.

Continue reading

8. March 2016

Music at home - HiFi gains a new source

Streaming music without corporate control allows you to listen to your music, not "consume" it

Self-sufficient hifi

Continue reading

7. March 2016

The Northern Lights

TinSlave-085624-07032016.jpg

Forget the science, just enjoy them

Continue reading

2. March 2016

The Superwind's first winter

Looks like the right kit

Continue reading

- page 1 of 3