Remote DBus continued (using your own program)
Filed under: Communications, Development, Linux
Continuing the previous thought on running DBus remotely using d-feet to check how it looks etc, this time, I wanted to call the DBus from my own program. Just write the DBus code as you would to query the DBus interface locally.
#!/usr/bin/env python import dbus import dbus.service from dbus.mainloop.glib import DBusGMainLoop BUS_NAME="org.freedesktop.DBus" OPATH="/" bus = dbus.SessionBus() obj = bus.get_object(BUS_NAME, OPATH) iface = dbus.Interface(obj, BUS_NAME) lala = iface.GetId() print lala
Then the magic comes in running the application.
DBUS_SESSION_BUS_ADDRESS="tcp:host=192.168.X.Y,port=Z" ./dbus-hello.py
Screenmovie 1.1 released
Filed under: Development, Frozentux.net, Linux, Ubuntu, Video
A quick note that Screenmovie 1.1 has been released. It’s still very crude, but adds some sound recording and the ability to turn it on/off. Postprocessing is not supported yet but should be there in the next version.
Features:
- Record video
- Record sound
- Configure file format
- Configure video codec + settings (5-6 codecs chosen for now)
- Configure audio codec + settings (2 codecs for now)
I still have some problems, but I just found some info on how to possibly make some problems better at least.
Todo:
- Fix some high cpu usage problems
- Add global keybindings
- Postprocessing encoding
- Clean up and add some values specific for codecs (as required).
Python multiprocessing
Sometimes I find a good time by writing small but easy to understand test programs to research specific behaviours in some language or another. Sometimes, the programs grows a bit wieldy and not so easy to understand, but all’s good that ends well. During the last year or so, I’ve grown more and more interested in python programming and finding it very enjoyable. There are some strange constructs that can be hard to wrap your head around, and sometimes I run into some very weird problems, but it’s not a big problem (imho).
My last forays has been into multiprocessing and how it behaves in Python. One of the features I had a hard time to wrap my head around since there are some syntactical weirdness that could use some addressing, or at least takes a bit of time to get your head around.
- Multiprocessing requires all objects to be pickled and sent over to the running process by a pipe. This requires all objects to be picklable, including the instance methods etc.
- Default implementation of pickling functions in python can’t handle instance methods, and hence some modifications needs to be done.
- Correct parameters must be passed to callbacks and functions, via the apply_async. Failing to do so causes very strange errors to be reported.
- Correct behaviour might be hard to predict since values are calculated at different times. This is especially true if your code has side effects.
The small but rather interesting testcode below explores and shows some of the interesting aspects mentioned above. Of special interest imho is the timing differences, it clearly shows what you get yourself into when doing multiprocessing.
#!/usr/bin/python
import multiprocessing
def _pickle_method(method):
func_name = method.im_func.__name__
obj = method.im_self
cls = method.im_class
return _unpickle_method, (func_name, obj, cls)
def _unpickle_method(func_name, obj, cls):
for cls in cls.mro():
try:
func = cls.__dict__[func_name]
except KeyError:
pass
else:
break
return func.__get__(obj, cls)
import copy_reg
import types
copy_reg.pickle(types.MethodType,
_pickle_method,
_unpickle_method)
class A:
def __init__(self):
print "A::__init__()"
self.weird = "weird"
class B(object):
def doAsync(self, lala):
print "B::doAsync()"
return lala**lala
def callBack(self, result):
print "B::callBack()"
self.a.weird="wherio"
print result
def __init__(self, myA):
print "B::__init__()"
self.a = myA
def callback(result):
print "callback result: " + str(result)
def func(x):
print "func"
return x**x
if __name__ == '__main__':
pool = multiprocessing.Pool(2)
a = A()
b = B(a)
print a.weird
print "Starting"
result1 = pool.apply_async(func, [4], callback=callback)
result2 = pool.apply_async(b.doAsync,
[8],
callback=b.callBack)
print a.weird
print "result1: " + str(result1.get())
print "result2: " + str(result2.get())
print a.weird
print "End"
The above code resulted in the following two runs, and if you look closely, the timing problems show up rather clearly. Things simply don’t happen in the order always expected when threading applications:
oan@laptop4:~$ ./multiprocessingtest.py A::__init__() B::__init__() weird Starting weird func B::doAsync() callback result: 256 B::callBack() 16777216result1: 256 result2: 16777216 wherio End oan@laptop4:~$ ./multiprocessingtest.py A::__init__() B::__init__() weird Starting weird func B::doAsync() callback result: 256 B::callBack() 16777216 result1: 256 result2: 16777216 wherio End
One more warning is in order. A job that leaves via the multiprocessing.Pool, and then calls the callback function has a major effect that could take some getting used to. The callback is run in such a fashion that if a class was changed, the change has not taken place in the context of the callback.
Screenmovie released
My first application ever in python has been released, Screenmovie as I chose to call it. It’s a very simple application for recording your desktop or windows on your desktop. Download and put somewhere in your filesystem and unpack it, and start it. It’s very simple to use, it uses pygtk to create a taskbar icon in your window manager. Click it once, and you get a cross-hair mouse pointer. Click the window you want to record, and the application starts recording straight off. Click the icon again to stop recording. To record the entire desktop, click the background somewhere.This application only works with Linux and other systems using X11 window management and requires xwininfo, ffmpeg and python 2.6 + pygtk+.
If you have any comments or thoughts, feel free to mail me.
Secondly, I’ve come quite far along with the pyEveApi, a rather large project I began a few weeks back. The project is perfect imho, it gives enough problems to solve and also introduces me to a lot of functionality in Python that I would have a hard time finding out about otherwise. I’m not close to a release yet as I only support 4 API’s so far, and just restarted quite a lot of the work with rewriting the caching functionality. The Long caching method has been fairly well implemented atm, and works, and I’m currently working on the Modified Long method, using the WalletTransactions API to test with. Either way, I’m hoping to have something releasable sometime soon.
Working with Python
A while back I started pondering Python as a language, and mostly the fact that I haven’t really tried any of the newer and more modern languages around. Not to any great lengths at least. So, about a year or two ago I started fiddling with it, mostly just trying different scripts, writing some small hacks, and so forth.First, I wrote a Bluetooth scanner for bluez, then via Dbus interface and so forth. The ease this came with was completely amazing tbh.
Last few weeks I started writing some larger applications with python, to try and learn it a bit better. First of all, I’ve had large problems with all desktop recording tools I’ve found so far — they either don’t support GL/Direct rendering, or they simply crash, or colors are all flunked out, and so forth. So, simple idea was to use the tool I know that works — ffmpeg and xwininfo — and make an easy to use interface for it, a taskbar icon. Click it once, and get a cursor to choose which window to record, click it and recording starts (icon changes to a recording button), and click it again to stop it. There is a right click menu to make some preferences and to stop the recorder as well. It works, and it’s very very simple :-).
Second set of tools I decided to make is a set of API’s for the Eve Api. This is also nothing new, but it was a good way to get a handle on how Python works, and how the Eve Api works. On top of this I have written some very simple apps that downloads all marketorders and wallettransactions in game and displays them nicely in a console. It has built in support of all caching styles, downloading, login, local disk cache, and so forth, but so far has very limited support for the different API’s.
My main observation so far has been that I like Python for it’s hackability and ease of use. Stuff comes naturally once you get used to it, but the dynamic typing and hackability also comes at some price. First of all, it is very easy to create scenarios where the code will crash due to bad types in the dynamic type checking, and the hackability also means you need to be very wary of your architecture. Hacking away is almost never a good thing if you are preparing to undertake any larger projects imho, the risk of writing bad code is paramount, and testing bad code … well, it’s just worse 😉 . That said, I do like the language, it’s completely awesome, and someone else said not so long ago, you can get around the problems, you just have to test the code so much harder for it to be reliable. Personally, I wouldn’t use Python for a critical system/application, but for my day to day use, it’s not a problem, I can survive a crash every once in a while. Also, after getting used to the pydev plugin to eclipse, I just love the entire environment even more :-).
Inproductive productivity
Filed under: Configuration Management, Linux, Personal, Projects, Windows
For a while I’ve been stuck in slow speed mode again, not really doing great work, just being on average. It feels weird. Don’t really get much done, but I have on the other hand had a great deal of time to test some “new” technologies, well, new as in only 10-15 years old I guess :-). I’ll get back to that later. Also, I’ve begun a new contract at “a big company”.
This is my first time at a really giant hunk of a company, the biggest I’ve seen before was circa 500 people in all, and it moved slower (the beaurocracy) than this in all honesty. This BigCompany is quite interesting to me. Started off with almost 4 weeks of introductions, courses, and so forth. They have a dedicated TEAM of CM’s, that alone is just… wow :-P. I’ve just been put up to speed and started working a little before this weekend so I might be a bit premature, but I like it so far. The weird part is, things happen, but not as I’m used to it. I’m used to 13+hour days and frentic coding/hacking to get things to happen, everyone here eschews away with their 8 hour days — only working overtime at very special occasions — yet slowly things get done, new functionality gets added and so forth.
Another thing that kind of amazes me — and worries me to some extent — is the kind of planning that is done. I’m used to small scale projects with workpackages or task based development, where no workpackage should ever take more than 4-5 days to implement. This place uses a workpackage development structure where each package takes up to 6-7 weeks for 6-10 people to implement. We’ll see how it works out — at least their “stand-up meetings” works :-).
All that being said, I had the time to write quite a bit of python which is a first, then I’ve looked into d-bus architecture which is also a first, and I also looked into Bluetooth and how to use it — some test applications running, fetching services and graphically displaying info about all units it finds etc. The complexity of Bluetooth is rather saddening imho, it’s a horrible protocolstack to work with in some senses, even though I was really impressed by how much python does for you.
I’ve been unused to the whole concept of python before this, and just a tad sceptical. Mainly because of all the problems with version matching that you always wind up having to do, to make anything work properly (try getting scons, trac and wamp, and some more tools working on a win32 machine some day for some fun).
Anyways, I always figured there has to be an upside, and there really is — python is hackfriendly 🙂 . In less than 3-4 hours I went from writing my first simple helloworld to having a scratch written class based graphical (tkinter) interface implementing some very fundamental bluetooth commands. In my world, thats not bad at all ;).
I’ve also had time to learn a lot of new tools at work. I’ll comment on those some other day as I havent seen much other comments on some of them (some is imho very expensive crap with a nice wrappings, while some are completely awesome). Sidenote, I simply adore the systems we are working on 4 xeon with 4cores and 64 gig ram.
I’ll get back later :-).
Trac on windows
Trac is a rightfull bitch to install on win32 as it requires very specific version matching of packages. This is a long winded installation note with the more or less latest versions available as of this writing.
I’m hoping to get working on TortoiseSVN and Trac integration once this is done (ie, make tortoise automatically set variables etc sent in change notes, which can hence update the issue tracking systems in trac and so forth. The way of working is really really sweet imho, and I think it could be a really nice way of working. I’m just sad that I have to set all this serverstuff up on windows though.
Let’s begin with a list of all the installation files used:
WampServer2-APACHE229.exe
svn-win32-1.5.2.zip
mod_python-3.3.1.win32-py2.5-Apache2.2.exe
These packages should be fairly simple to get started with. Install them straight on, in the order mentioned.
Unsupported version control system “svn”
For an Apache server here’s the essentials:1. Copy bin/mod_dav_svn.so and bin/mod_authz_svn.so to the Apache modules directory.2. Add the Subversion/bin directory to the SYSTEM PATH.3. Edit the Apache configuration file (httpd.conf) and make the following changes:3a. Uncomment the following two lines:
#LoadModule dav_fs_module modules/mod_dav_fs.so#LoadModule dav_module modules/mod_dav.so3b. Add the following two lines to the end of the LoadModule section:LoadModule dav_svn_module modules/mod_dav_svn.soLoadModule authz_svn_module modules/mod_authz_svn.so3c. Add the following to end of the file.<location /svn>DAV svnSVNPath c:\projects\svn\testAuthType BasicAuthName “My Subversion repository”AuthUserFile “c:\projects\svn\test\conf\passwd” Require valid-user</location>
<location /mpinfo>
</location>
<Location /trac>
</Location>
Added Miscellany section
A miscellany section has been added, containing lots of junk and other stuff that I am no longer developing, or that I have written previously. If anyone is interested in using any of the code, they are welcome to. This could mainly be looked upon as a junkyard of more or less useful scrap.
Miscellany update
Two new scripts added to the miscellany section, just for fun. I will add more later on, which may be a little bit more useful to you. Also, as some of you may have noticed, I have been very quiet for a long time. There are things happening that will take some time more to finish, that will hopefully make people a little bit happy in the future. As some has also noticed, there is the new banner on top of the page which points to a site I am partially related to working on. Hopefully, this will not cause too much of a problem for people:).