FreeNas, Transmission and Corrupted Files – Part 1

After many years of just ‘taking’ I mature and upgraded my setup to be able to give community back what I took.

I setup RAID 6 FreeNas machine. Thanks to playing with those I understood why everybody in FreeNas community hate Realtek and why people just say to throw it away and get something that works (FreeNas: Realtek 8111F and re0: Watchdog Timeout Error  and FreeNas: Realtek 8111F and “re0: Watchdog Timeout Error” – universal solution). In the end I join the ‘hate club’ and bought Intel PRO 1000 PT Dual Port NIC. Fun part is that Linux handle Realtek with ease but I been with Linux when It didn’t like most of hardware that Windows support so I can understand that there are still hardware issues with FreeBSD.

Cool, FreeNAS install without any issues.

If you unfamiliar with FreeNAS it comes with ‘plugins’ that are basically jails with pre-configured application that ‘just works’.

I slap that Install button and almost instantaneous I had working Transmission 2.93 (FreeNAS should work about pushing updates to plugins faster but yeah, nothing to complicate that we cannot fix ourself if needed)

I mounted needed directories, slap few torrent files that I already had there and enjoyed how transmission is handling all those sweet, sweet files out there.

And Tranmission was working and it was doing good. But after many, many torrent more that join the seeding happy farm not many people did connect to share the love from me. I started tweaking Transmission. Setting by setting I scope documentation looking for holy grail, some magic setting that would boost simultaneous connections. I found few. So I created new config file, and it was good. Not the best but I notice some connection increase. But I was thinking that maybe its because the application is inside jail aka virtual network interface. Few threads on popular forums, many many blog post later I saw the solution allow.raw_sockets=true. Great I saw a big improvement with connections – but who know if that was really all this tweaking doing or just someone join the swarm.

Few days later I notice big (I could even say tremendous) packet drop from/in my network. What was happening ? Router that is used between NAS and WAN have working QoS rules so it shouldn’t be a problem with using to much of bandwidth… So what was it ?

Culprit here was the CPU inside the router – we need to remember that using fancy feature that make router even smarter use CPU. QoS, Firewall, VPN all of them take some of that sweet-sweet pie (CPU). But why its dropping packets you ask? Simple, there is less and less CPU power to handle all the traffic that is going to and from network so everything is super slow. But wait, didn’t it work before Transmission? Yes it did – and we could disable some functions but we can do better. Lets optimize some firewall rules, we can cut QoS from 10 types to 5 without big sacrifices. Is that enough? No, but we can tweak both Transmission and firewall rule for it. Let set ‘Max connections’ to smaller number and double that with same (or 1% more on firewall rule so we are double sure that anything that is marked as ‘new connection’ and is about our limit get dropped).

And this game enough room for that sweet-sweet pie named CPU could handle ‘few’ packets more.

And it was good for months and months…

Until the corruption start to show !

(will continue in part 2)

Facebooktwittergoogle_plusreddit

MySQL row size too large (>8126)

This is something to do with bad table design, where you are using multiply columns with Text values, the best solution would be redesign tables, but from time to time you need to patch it at-hoc without redesigning application that use that database.

Solution that usually works:

 

Facebooktwittergoogle_plusreddit

Nagios – network/device configuration

If you don’t have working Nagios installation fell free to follow Nagios – New installation on Debian 9 link.

We will need to add folder structure (for better organization) to nagios config file:

Not everyone have luxury to always use smart switches around network – those make live a lot easier, but life is not a dream 🙂

Let us make a template for non-smart aka no-ping switches:

Now we can define few switches like this:

You can most variables put to template file so there will be even less writing

Most common type of templete is: linux-server (you can check others in /usr/local/nagios/etc/objects/templates.cfg )

After making few .cfg files inside our directory remamber to chown nagios:nagios -R /usr/local/nagios/etc/devices/ to avoid erros and restart nagios

Facebooktwittergoogle_plusreddit

How to setup VPN with PPTP

In the past there was a Personal VPN Server (OpenVPN) which gave us more security than PPTP could give us.

Then why would you want PPTP ? Because its faster and have lower footprint on CPU than OpenVPN. That way you can pick which you prefer in your case scenario. Also there is build-in support for PPTP in most devices.

Install:

And restart service:

Check if pptpd is listening to 1723 port:

Now setup some network stuffs:

Next in terminal:

Thats all on server side.

 

Client configuration:

Create config file:

Now we can connect to server (using name of the config file – pptpserver):

 

Facebooktwittergoogle_plusreddit

Personal VPN Server (OpenVPN)

Internet providers are collecting more and more data about our internet activities, but what can we do about it ?

You want to have secure access to your home servers, nas, devices ?

Virtual Private Network aka VPN is a solution for your needs!

 

1.Instalation

2.openvpn configuration

3.some system tweaks

4.Instalation and configuration firewall tool (so you don’t need to be iptables ninja)

5.generate CA, certs and keys for server

Just now we prepeared system envirement to generate, sign and distribute our certs thanks to CA (Certificate Authority).

Lets finish the fun with certs:

Move created certs and keys created for server:

 

and check if openvpn still starts (if not be sure there is no typo in config file or you moved correct files to correct location):

 

6.Creating certs and keys for clients:

Attention: user/group setting is not compatible with Windows

Client config file is still missing the paths for cert/key combo 🙂 but we will overcome this with one of two ways:

 

7a. Unified config file (one file to rule them all)

7b. Maybe you dont want to include cert inside profile file then we need to add this and copy needed files:

 

Facebooktwittergoogle_plusreddit

Running a secure DDNS service with BIND

If you lack of static ip for your home machines but would like to have static address to access them there is a simple setup to make on your primary DNS server to achieve that.

On our client side let’s generate some keys:

This will results in creating Kkeyname.***.key and Kkeyname.***.private, we will need to copy key from *.private file which will look like this:

Now on server let us edit BIND configuration file (eg. named.conf or named.conf.local) and add:

From now on we can allow zone updates by editing zone declaration and adding allow-update:

Last thing that we need to do on server i restart BIND

 

On client side we would need to use something capable of updating dns records (eg. nsupdate)

 

Facebooktwittergoogle_plusreddit

Fixing BIND’s journal out of sync with zone error

When BIND stop to work and throw error like this:

The simplest and fastest fix is by removing that journal file for zone

and then restart bind

 

 

Facebooktwittergoogle_plusreddit

Why I’m moving from Baikal to Radicale (CalDav & CardDav)

Saber/Dav which is a core component of Baikal is looking for maintainer (full story here)

It’s been some time now that I moved from Google Calendar service to self-hosted, open source solution.

I never notice any down sides with accessibility – maybe my need were simple enough 🙂

 

I’m not saying that Baikal is dead, if your are interested in Baikal fell free to lookup those posts (installation, updating)

 

As for me I started noticing one minor issue I had with Baikal, that is complicated calendar sharing setup – it IS possible to share calendars but its more complicated than it should be, it was announce that sharing will be added later after big code rewrite and I waited (because I wouldn’t even be able to contribute to php7.0+silex anyway), but here I found new solution… Radicale, it also don’t support sharing calendars via web-panel, but for me it got one BIG advanted in sharing zone, user rights, and with one little ACL list I was able to share whatever I wanted.

None of both solution support migration tools for this process so It was a bit of pain (as for now I wasn’t able to find too that would migrate my ‘Tasks’ from calendar, but I never used it as much and migration was done by hand with me clicking 2 buttons on each of 20 tasks I had left)

 

If you are interested, here are links related to process of installing, configure and migrate data from one solution to other:

  • Links coming soon
Facebooktwittergoogle_plusreddit

Creating swap partition

Swap partition is same thing as pagefile.sys for Windows. When we are using more RAM than we have, pagefile is used to store that extra data from memory.
For this Linux use swap partition. While you install Linux distribution most installers suggest to make swap partition that is twice the size of your RAM.
This is uncommon on VPS because that is a waste of storage on small servers.
But we can create and activate swap partition:

If Swap: 0 0 0 that means we aren’t using any swap partition.

Lets create and activate swap as root user:

Swap partition is use temporary, until we save information about it inside fstab:

Now we can enjoy swap even after reboot.

Facebooktwittergoogle_plusreddit

Pure-FTPd user management

Adding ftp account, which will use given os user:

 

Password change:

 

Removing ftp account:

 

Remamber, each command that change user information need to save those into proper database. You need execute this command to do it:

 

 

Facebooktwittergoogle_plusreddit