Screenmovie 1.1 released
Filed under: Development, Frozentux.net, Linux, Ubuntu, Video
A quick note that Screenmovie 1.1 has been released. It’s still very crude, but adds some sound recording and the ability to turn it on/off. Postprocessing is not supported yet but should be there in the next version.
Features:
- Record video
- Record sound
- Configure file format
- Configure video codec + settings (5-6 codecs chosen for now)
- Configure audio codec + settings (2 codecs for now)
I still have some problems, but I just found some info on how to possibly make some problems better at least.
Todo:
- Fix some high cpu usage problems
- Add global keybindings
- Postprocessing encoding
- Clean up and add some values specific for codecs (as required).
Python multiprocessing
Sometimes I find a good time by writing small but easy to understand test programs to research specific behaviours in some language or another. Sometimes, the programs grows a bit wieldy and not so easy to understand, but all’s good that ends well. During the last year or so, I’ve grown more and more interested in python programming and finding it very enjoyable. There are some strange constructs that can be hard to wrap your head around, and sometimes I run into some very weird problems, but it’s not a big problem (imho).
My last forays has been into multiprocessing and how it behaves in Python. One of the features I had a hard time to wrap my head around since there are some syntactical weirdness that could use some addressing, or at least takes a bit of time to get your head around.
- Multiprocessing requires all objects to be pickled and sent over to the running process by a pipe. This requires all objects to be picklable, including the instance methods etc.
- Default implementation of pickling functions in python can’t handle instance methods, and hence some modifications needs to be done.
- Correct parameters must be passed to callbacks and functions, via the apply_async. Failing to do so causes very strange errors to be reported.
- Correct behaviour might be hard to predict since values are calculated at different times. This is especially true if your code has side effects.
The small but rather interesting testcode below explores and shows some of the interesting aspects mentioned above. Of special interest imho is the timing differences, it clearly shows what you get yourself into when doing multiprocessing.
#!/usr/bin/python
import multiprocessing
def _pickle_method(method):
func_name = method.im_func.__name__
obj = method.im_self
cls = method.im_class
return _unpickle_method, (func_name, obj, cls)
def _unpickle_method(func_name, obj, cls):
for cls in cls.mro():
try:
func = cls.__dict__[func_name]
except KeyError:
pass
else:
break
return func.__get__(obj, cls)
import copy_reg
import types
copy_reg.pickle(types.MethodType,
_pickle_method,
_unpickle_method)
class A:
def __init__(self):
print "A::__init__()"
self.weird = "weird"
class B(object):
def doAsync(self, lala):
print "B::doAsync()"
return lala**lala
def callBack(self, result):
print "B::callBack()"
self.a.weird="wherio"
print result
def __init__(self, myA):
print "B::__init__()"
self.a = myA
def callback(result):
print "callback result: " + str(result)
def func(x):
print "func"
return x**x
if __name__ == '__main__':
pool = multiprocessing.Pool(2)
a = A()
b = B(a)
print a.weird
print "Starting"
result1 = pool.apply_async(func, [4], callback=callback)
result2 = pool.apply_async(b.doAsync,
[8],
callback=b.callBack)
print a.weird
print "result1: " + str(result1.get())
print "result2: " + str(result2.get())
print a.weird
print "End"
The above code resulted in the following two runs, and if you look closely, the timing problems show up rather clearly. Things simply don’t happen in the order always expected when threading applications:
oan@laptop4:~$ ./multiprocessingtest.py A::__init__() B::__init__() weird Starting weird func B::doAsync() callback result: 256 B::callBack() 16777216result1: 256 result2: 16777216 wherio End oan@laptop4:~$ ./multiprocessingtest.py A::__init__() B::__init__() weird Starting weird func B::doAsync() callback result: 256 B::callBack() 16777216 result1: 256 result2: 16777216 wherio End
One more warning is in order. A job that leaves via the multiprocessing.Pool, and then calls the callback function has a major effect that could take some getting used to. The callback is run in such a fashion that if a class was changed, the change has not taken place in the context of the callback.
Regarding web browsers and User Interfaces
Filed under: Development, Frozentux.net, Linux, Windows
I had a discussion this morning with my wife regarding her new computer (Toshiba laptop). A few days back, I was contemplating the possibility to try and get rid of the Windows 7 license it came with the EULA disagreement argument and get my money back, but after much consideration and realizing that her school (Chalmers University) pretty much requires her to run Windows 7 and Office 2007 (all applications used are windows based with Unix/Linux alternatives only irregularly, and her lab professors more or less frowns on OpenOffice.org vs MS Office formatting problems), we decided to keep it. This is another story, how Chalmers went from being a very open University to a complete clusterfuck of a Microsoft bootcamp and something I’d love to write more about at some point, but not now.
The computer has arrived finally, and my wife had a little time to play around with Windows 7. I’ve personally not used Windows since Win XP and refuse to lay hands on it unless “forced” to at work. Her main gripes came with Internet Explorer 8, and I couldn’t help but laugh very loudly at her assessment of it. It incessantly keeps asking questions and giving hints and ideas like “How do you like this?”, “Did you know that IE plugin blah …?”, “We can help you do …?”, “Do you have herpes?” to the point where the only sane comparison she could make is a very horny and needy drunk boy at a club. It took her less than one battery charge to switch back over to Firefox. My guess is that they simply made the user interface a bit … too helpful, and tried to be too reactive to “possible needs in this context”.
So, on this topic, I thought it could be slightly interesting to see this months statistics and split between different operating systems and web browsers.
Operating systems
I can’t really say that this site is representative of the world in general, but I think it is interesting to see the level of windows users. I’m guessing this is a sign that many of the people visiting are reading about iptables and using linux for servers and firewalls, but aren’t using it as a desktop system (yet?).
| Windows | 59.1 % |
| Linux | 31.3 % |
| Macintosh | 6.4 % |
| Unknown | 2.7 % |
Web browser
Personally, I find this part the most astounding. That IE has gone so low is simply not something I had expected, and how big market share firefox has… incredible.
| Firefox | 60.7 % |
| MS Internet Explorer | 16.4 % |
| Safari | 11.4 % |
| Mozilla | 4.5 % |
| Opera | 4.1 % |
| Unknown | 1.2 % |
I will try to do some more work on this in the future, could be interesting to see this change over time. I know the reliability of my own source isn’t excellent and the error margin is probablythrough the roof, but it should be rather fun to dig in to.
Screenmovie released
My first application ever in python has been released, Screenmovie as I chose to call it. It’s a very simple application for recording your desktop or windows on your desktop. Download and put somewhere in your filesystem and unpack it, and start it. It’s very simple to use, it uses pygtk to create a taskbar icon in your window manager. Click it once, and you get a cross-hair mouse pointer. Click the window you want to record, and the application starts recording straight off. Click the icon again to stop recording. To record the entire desktop, click the background somewhere.This application only works with Linux and other systems using X11 window management and requires xwininfo, ffmpeg and python 2.6 + pygtk+.
If you have any comments or thoughts, feel free to mail me.
Secondly, I’ve come quite far along with the pyEveApi, a rather large project I began a few weeks back. The project is perfect imho, it gives enough problems to solve and also introduces me to a lot of functionality in Python that I would have a hard time finding out about otherwise. I’m not close to a release yet as I only support 4 API’s so far, and just restarted quite a lot of the work with rewriting the caching functionality. The Long caching method has been fairly well implemented atm, and works, and I’m currently working on the Modified Long method, using the WalletTransactions API to test with. Either way, I’m hoping to have something releasable sometime soon.
Working with Python
A while back I started pondering Python as a language, and mostly the fact that I haven’t really tried any of the newer and more modern languages around. Not to any great lengths at least. So, about a year or two ago I started fiddling with it, mostly just trying different scripts, writing some small hacks, and so forth.First, I wrote a Bluetooth scanner for bluez, then via Dbus interface and so forth. The ease this came with was completely amazing tbh.
Last few weeks I started writing some larger applications with python, to try and learn it a bit better. First of all, I’ve had large problems with all desktop recording tools I’ve found so far — they either don’t support GL/Direct rendering, or they simply crash, or colors are all flunked out, and so forth. So, simple idea was to use the tool I know that works — ffmpeg and xwininfo — and make an easy to use interface for it, a taskbar icon. Click it once, and get a cursor to choose which window to record, click it and recording starts (icon changes to a recording button), and click it again to stop it. There is a right click menu to make some preferences and to stop the recorder as well. It works, and it’s very very simple :-).
Second set of tools I decided to make is a set of API’s for the Eve Api. This is also nothing new, but it was a good way to get a handle on how Python works, and how the Eve Api works. On top of this I have written some very simple apps that downloads all marketorders and wallettransactions in game and displays them nicely in a console. It has built in support of all caching styles, downloading, login, local disk cache, and so forth, but so far has very limited support for the different API’s.
My main observation so far has been that I like Python for it’s hackability and ease of use. Stuff comes naturally once you get used to it, but the dynamic typing and hackability also comes at some price. First of all, it is very easy to create scenarios where the code will crash due to bad types in the dynamic type checking, and the hackability also means you need to be very wary of your architecture. Hacking away is almost never a good thing if you are preparing to undertake any larger projects imho, the risk of writing bad code is paramount, and testing bad code … well, it’s just worse 😉 . That said, I do like the language, it’s completely awesome, and someone else said not so long ago, you can get around the problems, you just have to test the code so much harder for it to be reliable. Personally, I wouldn’t use Python for a critical system/application, but for my day to day use, it’s not a problem, I can survive a crash every once in a while. Also, after getting used to the pydev plugin to eclipse, I just love the entire environment even more :-).
Final thoughts on the embedded Linux seminar
The embedded Linux seminar was held last week, and in general I feel that it went pretty well in line with my expectations. It’s been a long time since I held any real presentations, so the first two presentations in Gothenburg I was very much nervous and lost myself a few times along the line. The two presentations in Stockholm and Oslo went surprisingly well however, and I don’t think I made any huge errors from a pure presentation point of view.
Licensing
Now, that said, we did run into a few snags and after pondering what we can learn from the whole presentation/seminar — and there are a few points I’d like to raise both for the attendees and anyone else who might be interested. Being an “engineer”, I like to consider what went wrong, etc. Most points are of minor interest, but one of the absolute major points that we really didn’t get across properly, licensing issues with open source, or rather, licensing issues are manageable, however this was not the main area that we (or at least me) where there to talk about. Our main error was simply that we forgot communicating properly with each other, and correlate what we where saying. Also, this seminar wasn’t really about licensing issues, Nohau has an entire seminar/course on that topic alone, and you could easily fill out an entire university course on open source licensing.
The main point I tried to get across was that, yes, you need to be wary about licenses and you need to look at what is required of you, but that’s nothing different from any closed source licenses either, and you should be putting policies as well as processes in place to handle it, and push knowledge on how to handle licenses must be disseminated throughout the project.
So, to address some of the main licensing questions we received:
- No you will not have to give away your code if you link the code properly.
- You will have to set up proper procedures to handle any third party sources.
- You will have to create processes for everyone to follow to get any third party sources “accepted”.
- You will have to adhere to third party licenses, if you don’t, be prepared to be forced to and receive some bad publicity for it. (A lot of companies/people do get away with it, but is it worth risking it?)
Reliability
The second large question we got was, when would you use Linux, and when wouldn’t you use it, in a life or death situation? Simply put, I wouldn’t put it in a system where a person or persons would die if the process/hardware/appliance crashes, but that’s me. I would generally speaking make the life supporting/critical system run on a separate hardware, and then make all the critical stuff run in that context/hardware, and then a second piece which communicates with the critical hardware and do the higher end “stuff” that might be interesting (communicate to centrals, user interfaces, settings, etc). This way, the critical stuff can be kept simplistic and reliable (in my experience, reliability is a function of complexity, the more complex, the higher the failure rate).
In most projects, this has to be decided on a case by case basis, and due diligence must most of the time be taken with the laws and standards of each area. What is possible and advisable to do in house and home automation is not the same as in airplanes or medical systems for pretty obvious reasons.
Presentation depth/breadth/focus
Finally, a minor point, I got some criticism for being too shallow, not going enough in depth. Well, I could have stayed on discussing tool-chains and how to make one for hours, or I could have talked entirely about Linux internal boot order and why it works the way it does, but that wasn’t the goal of the entire seminar. That stuff could be studied to death, in the end your better off “getting” the top-down structure of an embedded Linux project and then just experiment on your own rather than get everything served in forum that can not make justice to everyones requirements. Next time however, I will try to maintain a deeper focus on a bit fewer topics, or get more time to speak in.
Anyways, I think it was fun and a huge experience. I hope most people visiting found the seminar interesting and had something out of it.
Using Ubuntu as Media Server for Xbox360
I got a Xbox360 since about a year and I just noticed it had some way of connecting to a PC, using the PC as a Media Server. Unfortunately it required a Windows Media Center installation to work, or so it claimed at least. This is probably not news to anyone, but it was very easy to get Ubuntu (or any other Linux distro as a matter of fact) to serve media for the Xbox 360. Xbox 360 uses UPnP to get media from the Windows Media Center PC. To make any recent Ubuntu able to serve UPnP suitable for the Xbox, do the following:
- sudo apt-get install ushare
- sudo dpkg-reconfigure ushare
- sudo vim /etc/ushare.conf
- Make sure all the settings are correct.
- sudo vim /etc/default/ushare
- Make sure it contains USHARE_OPTIONS=”–xbox”.
- sudo /etc/init.d/ushare restart
You should now be able to find the PC by searching for it from the Xbox interface (the name you set in ushare.conf should show up in the list of found PC’s). Now that that’s said, I should hint that the Xbox360 has a really shitty availability of audio and video codecs, and I don’t know if it’s possible to resolve this problem. There are hints that there is something called a UPnP Media Adaptor on the ushare website which should be able to convert to proper file formats as necessary, but ushare does not have that ability. Of course, that would give a shitload of cpu load on the fileserver as well, something which sounds less good in my opinion.
My personal opinion so far, Xbox 360 media center is really simple to use, but the available codecs, flexibility and scalability is catastrophically bad in comparison to my MythBuntu installation (still running 9.04 though). The Mythbuntu installation is a bit heavy on the configuration, but much more flexible, handles almost all codecs I’ve run into without even a hitch, and very scalable.
embedded Linux seminar 25-27/11 2009
On the 25-27 of november I will be on tour with nohau.se and hold a embedded linux seminar. The entrance is free, but requires a registration, see the embedded linux seminar webpage. according to the following schedule:
- Göteborg 25/11, em 13-16. Plats: Centralhuset Konferens, lokal Orientkusten
- Stockholm 26/11, em 13-16. Plats: Kista Konferens, lokal Alfa
- Oslo 27/11, fm 8-12. Plats: Thon Hotel Vika Atrium, Munkedamsveien 45, Oslo
I will specifically do the Development using Embedded Linux track, which will be 50 minutes long. The presentation is still fairly crude and rough around the edges, but some of the bulletpoints I’m going to talk about is:
- When to use Linux/ When NOT to use Linux
- Pitfalls of open source vs closed source and vice versa
- Hardware vs Development time cost decisions
- Choosing the right hardware
- Choosing the right software
- Security.
I hope to see some of you at the seminars!
Example transforming videos
I recently made some video editing on videos i copied to and from my cell phone and realized some of the stuff might be rather esoteric and hard to find good examples on how to do. Basically just going to post some minor tips and tricks that I picked up, and some very simple commands to use with mencoder, ffmpeg and kino.
I used mencoder and ffmpeg to do some of the basic edits, like turning videos around etc. After the basic video snippets where done, throw them into kino and make the final cut, and then recode the video into a distributable format (10 minute video in dv format as used in kino = 2.1gig data, while 10minute divx of the same video = 170 meg).
#Rotate video 90degrees
oan@laptop4:~$ mencoder -o lala.avi -vf-add rotate=1 V170709_12.54.AVI -oac copy -ovc lavc#Postprocessing filters, ac = high quality
oan@laptop4:~/Desktop$ mencoder -o lala.avi -vf pp=ac V170709_12.54-recode.AVI -oac copy -ovc lavc#Transcode video so it works on cellphone (KC910), this “works for me”(tm)
oan@laptop4:~/Videos$ mencoder -o lala.avi -oac copy -ovc lavc -lavcopts vcodec=mpeg4 alice-final.avi# Create a black 2 second frame (25 fps, 50 frames), I used this as a filler between
# two movies. There’s probably easier ways of doing this, but it “works for me”(tm)
oan@laptop4:~/Pictures/2009-08-01$ ffmpeg -r 25 -loop_input -i black.jpg -vcodec mjpeg -vframes 50 -y -an test.avi
Finally I put the videos together in kino in the order I wanted, with black frames in between and effects fading from the videos into black, and so forth, making for smooth transitions.
Dpkg and apt-get reading database is really slow [fixed]
I’ve had problems many times with the “reading database” step in deb package based systems getting really slow. Today it took over 1,5 minute on a quad core machine with 4 gig ram, which is simply over the top for me.
Within 5 minutes of searching I found this:
http://antti-juhani.kaijanaho.fi/newblog/archives/521
So a total of 7 minutes later, it takes me less than 2 seconds to install small packages again. I should’ve found this earlier :-).