Iptables-tutorial and ipsysctl-tutorial on github

I guess I should have done this a long long time ago. Both the iptables-tutorial and the ipsysctl-tutorial source code are now available on github. Many many years ago I had an idea of putting the version control system out there for use, but I never realized it for various reasons. Both these documents are old by today, but the basic information is still valid and will likely be for a long time to come it seems.

I apologize for the version history, I moved from CVS in a rather rude way to SVN without keeping the history, which was what I used back in those days.

I invite anyone and everyone to do edits if they wish to and send me pull requests to fix the issues they find, or to add the documentation they’d like to add.

The iptables tutorial is available at:
https://github.com/frznlogic/iptables-tutorial

The ipsysctl tutorial is available at:
https://github.com/frznlogic/ipsysctl-tutorial

Project build speed importance

I began writing this several years ago but never published it for various reasons. I think some of the thoughts are still really interesting, I joined a company named Pelagicore some months ago and we do some rather large builds at this company, similar in build times in the project I refer to in this text, but with some major differences. This build is based on yocto and it actually works really well once you get through the first build (currently a scratch dev image build is up to 4 hours in time, but after that it’s able to rebuild only changed software and recipes as necessary. The scratch build is heavy admittedly, but we are working on some ways to improve that as well, such as sstate caches, icecc distributed build systems, and so forth. I think with modifications we could get those 4 hours down to under an hour, which would be really sweet. Anyways, I thought I’d post this text as is, even though it’s not finished, just because and since I re-read it right now 😉 .

Original text


Again, I’m struck by the importance of a proper build process in projects. No matter how small or big the project is, the build must be kept fast and working from get go to the end — which means its end of life cycle. I am currently working in a large project with a huge code base where the build system is, in my opinion, next to completely collapsed. Build time is in excess of 1-4 hours, and the automated test-suites take from 3 hours to 7 hours to complete, depending on how thorough a test-suite you choose. A gigantic heap of time is spent just…. waiting… waiting…. waiting. Just to make a simple one line edit and then compile to see if it compiles can take up to 4 hours. I’ve previously been a bit spoiled with good and simple build systems, only occasionally running into really crappy build systems — for some reason, a lot of scientific open source software seem to be stuck in this category.

This time, I think I hit pay-dirt on “how not to do it”. Instead of focusing on the bad parts, I will try to focus on the things to do and to keep in mind. Nobody seems to like a grumpy person anyways, which I really am sometimes.

1. Keep the build system simple and manageable. Try to maintain the build system in a logical fashion and in a single language/system (scons/python, Makefiles, etc).
2. Expandable (new directories, files, etc)
3. Scalable (multiple CPU’s, machines, threads, etc)
4. Try to use as few frontends as possible (a single top makefile for example, with targets depending on what you want to do), and keep people informed of changes. Wiki with history via RSS is _perfect_ for this. This doesn’t mean you can create a script to build something, then 2 months down the road delete it because it is no longer needed, it causes a lot of stress for developers who just barely had a chance to find the script.
5. The programmers are (most likely) your customers.

DIY OSD update

December 11, 2014 by · Leave a Comment
Filed under: Development, FPV, Hardware, Robots, Video 

Late update, but video is working as expected, GPS is working to some extent, but because of local weather I havent really tested it outdoors… I’m considering changing video transmitter and power system however. I’m currently running on 1S 160mAh batteries with a stepup regulator to 5v to a 10mw transmitter, which gives a really lousy battery time for the video system, also I’ve got too many batteries flying around at the moment I think.. Limiting to a bit fewer might be a good idea. Currently, there’s a battery for the video link/osd, one built into the camera, one for motor/rc receiver/servos, and one in the rc transmitter, 4 batteries in total of various sizes etc… a pain to charge them all. Hooking up the videolink and OSD to ESC/BEC is a trivial change and would remove a lot of cruft (stepup module etc), but will also add the EMC from motor/ESC to handle for the videolink etc as it’s a completely separate system today.

I’ve also upgraded so I’m using a mobius actioncam. Everything all in all means the system will draw more power and weigh more as well. Removing some batteries and running from the mains system would be really nice.

qmake project custom install and files being installed as non-executable

December 4, 2014 by · Leave a Comment
Filed under: Development, Linux 

I had some issues with creating a custom install target from a qmake pro file for a project today and everything took me quite some time to get working I’m embarassed to say. This was done to create make install_ptest target for a yocto build, but the files kept turning up non-executable in the rpm’s and install directory. In the end, I got it working by adding the executable keyword to the CONFIG of the target, in addition to the no_check_exist.

PTEST_FILES += \
    $$ROOT_SOURCE_DIR/tests/service_tests/test_runner.py \
    $$ROOT_BUILD_DIR/tests/service_tests/tst_service_tests

ptest.depends += $(first) sub-tests tests
ptest.path = /usr/lib/AnyApp1/ptest
ptest.files = $$PTEST_FILES
ptest.CONFIG = no_check_exist executable
INSTALLS += ptest
QMAKE_EXTRA_TARGET += ptest

I hope this helps you if you run into the same problem as I had some issues finding information about that specific flag or the problem.

Build ppa package

November 26, 2014 by · Leave a Comment
Filed under: Debian, Development, Linux, Ubuntu 

To build a package for ppa distribution, you need some tools. To “cross compile” for releases, for example i386 and amd64 packages on the same machine, takes some more work with schroot, dchroot etc. I’ll start with explaining how to create a “local” package for your own host, I’ll add another entry on how to do an i386 package from amd64. Everything is done on ubuntu 14.04 amd64 machine in this case, and I’m rebuilding dbus.

In short you need:

  1. apt-get install build-essentials dpkg-buildroot schroot gpg
  2. gpg –gen-key
  3. apt-get build-dep dbus
  4. mkdir dbus-amd64 && cd dbus-amd64
  5. apt-get source dbus
  6. export DEB_SIGN_KEYID=
  7. cd dbus-directory
  8. make changes.
  9. dch -i
  10. dpkg-source –commit
  11. dpkg-buildroot -i -I

If you plan on publishing your deb packages to launchpad or some such, you need to create an account and add a ppa. This is simple and done via the http://www.launchpad.net webpage. The webpage also gives you good upload information. Note that they require signed files, so signing must work for you first.

8. Create account on launchpad.
9. Export the gpg generated key to hkp://keyserver.ubuntu.com:11371 (easiest to do via Passwords and Keys tool
10. Import the key to launchpad using the key fingerprint.
11. Create a new PPA from the launchpad dashboard
12. dput ppa: dbus_1.6.18-0ubuntu4.4_source.changes

The package will be built by launchpad on its own, this may take some time..

DIY OSD

November 10, 2014 by · Leave a Comment
Filed under: Development, Robots 

I’ve built a DIY OSD based on Dennis Frie hardware and software released on rcgroups forum. I’m fairly happy with the results, the osd actually work. I do have some issues, but nothing serious. I have GPS, voltage sensors working as they should so far and am planning to add a current sensor and possibly rssi as well.

image

Top side not much to talk about. It’s an arduino pro mini 5v unit.

image

The GPS in the picture is not the one I’m using so far, but I will move over to it in a few days I hope, if I find the time. Main problem is that the GPS requires 3.3v,I’m running everything on 5v at the moment. I’m currently using a ublox neo7m from banggood. The picture shows how I hope to mount the GPS approximately though, depending on how I plan to mount OSD in comparison to the video link and the camera… GPS and video link should be mounted as far away from each other as possible, but OSD needs to be close to the video cable.

image

The bottom side. Most of the components have been sandwiched between the PCB’s. Some of the routing is a bit stupid but I’ll see if I can fix it so I can fit the current sensor somewhere on the bottom as well.

Remote DBus continued (using your own program)

November 7, 2014 by · Leave a Comment
Filed under: Communications, Development, Linux 

Continuing the previous thought on running DBus remotely using d-feet to check how it looks etc, this time, I wanted to call the DBus from my own program. Just write the DBus code as you would to query the DBus interface locally.

#!/usr/bin/env python

import dbus
import dbus.service
from dbus.mainloop.glib import DBusGMainLoop

BUS_NAME="org.freedesktop.DBus"
OPATH="/"

bus = dbus.SessionBus()
obj = bus.get_object(BUS_NAME, OPATH)
iface = dbus.Interface(obj, BUS_NAME)
lala = iface.GetId()
print lala

Then the magic comes in running the application.

DBUS_SESSION_BUS_ADDRESS="tcp:host=192.168.X.Y,port=Z" ./dbus-hello.py

DBus remote connection

November 5, 2014 by · Leave a Comment
Filed under: Communications, Development, General, Linux 

In a project I’m working on at the moment we wanted to remotely monitor a DBus session bus. The system in question has several buses available using different users and systemd. d-feet can connect and monitor remotely via TCP.  The following code will allow you to connect to a remote DBus on a target development board for example.

First copy the original session.conf to separate configuration files for each user.

cp /etc/dbus-1/session.conf /etc/dbus-1/session.conf.<username>
cp /etc/dbus-1/session.conf /etc/dbus-1/session.conf.<username2>

Then for each of the newly created configuration file, add the following configuration but with different port numbers and correct username director in /run/user. The ip address should be the IP of the connecting host, not the server. Edit session.conf.<username> and add:

<listen>tcp:host=<ip>,bind=*,port=<port>,family=ipv4</listen>
<listen>unix:path=/run/user/<username>/dbus/user_bus_socket</listen>
<listen>unix:tmpdir=/tmp</listen>

<auth>ANONYMOUS</auth>
<allow_anonymous/>

The systemd script is rewritten to use a specific conf file for the specific user trying to start the DBus.

Edit  /lib/systemd/system/dbus-session@.service and rewrite the ExecStart line as follows.

ExecStart=/usr/bin/dbus-daemon --config-file /etc/dbus-1/session.conf.%i --nofork

This allows you to connect using d-feet or other dbus applications (potentially, you should be able to connect for example other services over the network to the new DBus….).

Choose “connect to other bus” and use as bus address:

tcp:host=<targetIp>,port=<port>

Done. Hopefully.

Testing out RasPlex

November 2, 2014 by · Leave a Comment
Filed under: Communications, Linux 

I decided to try out RasPlex after having run OpenElec and RaspBMC for about half a year, and before that XBMCbuntu for a few years. I’ve been using plex server on my file server for a few months already and plex on the tablets/phones to stream movies, tv shows, etc while in bed. I haven’t had that much time with RasPlex yet, but I’m really impressed so far, especially when you start looking at what type of infrastructure it enables. I’m now using the same chain of applications for my tv, phones, tablets, and laptops, this is something I’ve never been able to before (if you except NFS/SMB solutions with VLC or some such video player, which has proven unstable for the phone/tablet cases in my case, high def videos will stutter a lot among other things).

My main problem with the entire setup is that my fileserver is a bit too weak to transcode 1080p on the fly, it is a D525 Atom CPU however, and I didn’t see this usecase when I bought it, so I’ll have to live with it for now ;).

All that said, I’m still very happy with XBMC, but I feel that RasPlex so far has given a more… mature feeling. It seems to work very well with no big hickups so far, the UI on RaspBMC and OpenElec was really slow, something that still holds true in RasPlex, but it is better.

Gps issues

November 2, 2014 by · Leave a Comment
Filed under: Communications, Development, Linux 

Been working for a few days with three different GPS modules (different builds etc) and I’m getting the same problem with all of them. They are pretty much giving out complete garbage at all settings. At 9600 bps 8n1 it gives out patterns at least. Just unreadable. I thought it might be ubx protocol but probably not.

I feel like I’m doing something really stupid but I can’t figure it out. Serial port problem? What else could be wrong?

« Previous PageNext Page »