Sunday, 22 April 2012
Avoiding cyber squatting failure
A year or two ago I misspelled a domain name of very popular site used by many developers. It came up as "Not Found" so I realised as the site at the time was still quite niche (not any more) cyber-squatters had not cottoned on to it yet.
So I registered a couple of similar domain names with a misspelled vowel. This was mainly as I thought it was quite funny at the time, but also I did not want real professional & cynical cyber-squatters to register them either.
Of course I did not know quite what to do with them so I put them as Google parked domains (AdSense for Domains) and forwarded all emails automatically to the proper domain. If ever contacted by the proper site I would just let them have the domain(s).
End of AdSense for Domains
This Spring Google closed their AdSense backed parked domain offering. And I needed to reflect what to do with the domain names and what options I have.
I do not really want to keep paying for their registration, the ads on them brought in less than £20 every year so not likely to challenge my ethical backbone either.
When some of the domain names expired this Spring, and I was initially just going to let them expire, but then from experience I know domain cyber-squatters will be scanning for expired domain names and pick them up. And they will use them far more cynically and unlikely to ever hand them over for proper usage. So I extended my registrations with those domain names.
Domain parking bad taste
In the end I moved the domains from AdSense For Domains to another Domain Parker.
But this has left a really bad taste in my mouth. I am not really any better than a cyber-squatter. I am profiting from misspellings, although less than the registration costs... The content on the parked domains are really of no assistance to the people trying to reach the proper site. So I cannot leave it as it is.
"Nice" Anti Cyber Squatting
The best solution would be if there was a friendly not-for-profit community back anti-cyber-squatter service. Offering useful content/redirection. A sort of defensive registrations the community can do on behalf of proper sites. Naturally the proper sites should really register these names themselves, but some cant or wont. But to avoid squatters taking advantage a free service like this would be handy. Of course we/I would still have to pick up the registration cost.
Landing page
But instead I think I might put up a brief comical landing page with a big button to go to the proper site. (However until I actually get round to create that, the domains are still listed with the domain parker.... )
Wednesday, 22 February 2012
Clone local git repository to remote server?
If you have a local repository that you want to clone/copy to a remote server here is how I do it.
Perhaps you have been scaffolding, testing and initial silver bullet development and have realised the project is mature enough to share with others. Or you just want it backed up remotely.
Simple
What you need to do is on the remote server create a bare git repository:
Then add this remote to your local git repository:
Now every time you want to push to your remote server, you do a normal push:
Although I would seriously consider using something github or gitorious instead of your own server.
Perhaps you have been scaffolding, testing and initial silver bullet development and have realised the project is mature enough to share with others. Or you just want it backed up remotely.
Simple
git clone local remote do not work. Destination path for git clone is always local.What you need to do is on the remote server create a bare git repository:
remoteserver$ mkdir -p /var/local/repos/myproject;
remoteserver$ cd /var/local/repos/myproject;
remoteserver$ git init --bare;Then add this remote to your local git repository:
localmachine$ git remote add origin \
ssh://remoteserver.example.com/var/local/repos/myproject;Now every time you want to push to your remote server, you do a normal push:
localmachine$ git push origin master;Although I would seriously consider using something github or gitorious instead of your own server.
Monday, 13 February 2012
Continuous Deployment via Stack Overflow
Etiketter:
agile,
architecture,
code,
productiviy
I do have an interest in Continuous Deployment/Continuous Delivery. Continuous Deployment goes beyond Continuous Integration and automatically deploys builds to production all the time, not once in awhile. It goes well with agile thinking.
With frequent deployments the delta that can go wrong is very small, the feedback is very quick and the knowledge is fresh and active with the developers so fix turnarounds are immediate. By relying heavily on automated integration testing and DevOps that provide automated deployment the result is quick and painless.
This week I answered a difficult question on Stack Overflow. The question by Emwee was for advice on how to use Continuous Deployment with multitude of inter-dependant systems.
It is a tricky question and I did not have an exact answer. My answer was more along the line of how to facilitate an easier deployment by making the dependency and coupling looser and roll-outs smoother.
I referred to Duck Typing, Feature toggles, evolution scripts, version tables, symlink stage & deploy, and referenced a Hacker News discussion on how Amazon deploys their systems, a video on how Netflix builds releases in the cloud and how at IMVU they deploy 50 times a day.
I also referred to Humble and Farley's book on Continuous Delivery.
There was another reply as well by EricMinick referring to his previous answer to a similar question. Eric does in detail describe scenarios of using promotion builds into different isolated test environments and suggest solutions that his company UrbanCode provides in relation with Jenkins.
In the end Continuous Deployment is a great evolution. But with large enterprises you need to keep your tongue straight. But it is worth the investment.
With frequent deployments the delta that can go wrong is very small, the feedback is very quick and the knowledge is fresh and active with the developers so fix turnarounds are immediate. By relying heavily on automated integration testing and DevOps that provide automated deployment the result is quick and painless.
This week I answered a difficult question on Stack Overflow. The question by Emwee was for advice on how to use Continuous Deployment with multitude of inter-dependant systems.
It is a tricky question and I did not have an exact answer. My answer was more along the line of how to facilitate an easier deployment by making the dependency and coupling looser and roll-outs smoother.
I referred to Duck Typing, Feature toggles, evolution scripts, version tables, symlink stage & deploy, and referenced a Hacker News discussion on how Amazon deploys their systems, a video on how Netflix builds releases in the cloud and how at IMVU they deploy 50 times a day.
I also referred to Humble and Farley's book on Continuous Delivery.
There was another reply as well by EricMinick referring to his previous answer to a similar question. Eric does in detail describe scenarios of using promotion builds into different isolated test environments and suggest solutions that his company UrbanCode provides in relation with Jenkins.
In the end Continuous Deployment is a great evolution. But with large enterprises you need to keep your tongue straight. But it is worth the investment.
Friday, 3 February 2012
Multi book reading
I have a habit that I wonder if others do as well. And I wonder if it is productive or counter intuitive.
What I do is multi-book-reading. By that I mean I read several books at once.
How I do this is by having different books in different locations. And also several books in the same location. I have a preference for printed books. PDFs of books are handy occasionally for specific searches, but I really cannot read page upon page on a screen. Kindle may be nice, but I have not yet jumped on that bandwagon.
This multi-book effect is partially due to laziness, as when I am in the lounge I do not want to have to walk back upstairs to my study if I get a sudden urge to read or fetch a book. So I always have a half read book in the lounge. (Currently in the lounge I am re-reading the Kanban book by Anderson).
In my study I naturally have bookshelves loaded with computers books, especially java, and always a couple on my desk. I tend not to read too long in this room (then why is it called a study?), but a quick few paragraph while the pc is rebooting etc a few times a day. The reference books however might be referred to when needed. (Current study books are Seven Languages in Seven Weeks by Tate and Continuous Delivery by Humble & Farley)
On my bed side table there may be a fiction book. I don't really want to think too much when trying to sleep.
At work I would also have a mini library as I also tend to evangelise and lend books to colleagues. So I usually have a half read book or two or three there as well. Again for quick glances while rebooting etc, but also for longer reads during lunch if I go by myself. (Just finished ReWork by Fried & Heinemeier Hansson and Linchpin by Godin)
When I am out-and-about e.g. queuing for the till in a shop, waiting for the bus or at lunch when I forgot a book, I use the Aldiko e-book reader on my Android phone. Being a sucker for offers at O'Reilly for epub formatted books I have quite a few on my phone. Aldiko is excellent, but the small phone format is in essence rubbish for reading books over time. It is good for quick 1-5 minutes reads but no longer. And useless for reference books as the overview of the page is difficult. (97 Things Every Programmer Should Know by Henney is good for 5 minute reads)
For longer commutes by train or plane I tend to bring an actual book. If I do this regularly I should probably invest in a Kindle. (Read bits of Specification by Example by Adzic on the plane recently)
I tend do a lot of quick reads. As a stop gap filler between other events. Maybe work interrupts, maybe it is some element of ADHD, or probably our 6 months old daughter. This means I constantly waste time remembering the context of where I am in the book.
Longer reading periods is usually a holiday pattern. This would then often involve more fiction books. I read the Night's Dawn trilogy[2][3] over the last few holidays.
Some books I might skim read chapters to get a gist of but I know I can revisit them if I need to know those sections in detail in the future. Some books I never finish. I would say a good third or more in library I have not finished or barely read. Some turned out just not interesting, some are just skim read while others are consequences of multi-book-reading and lower down the reading priority list.
So this multi-book-reading means I do read a lot of books. But it also means I do not finish enough books. Not very "kanban" or "one-piece-flow" :)
What I do is multi-book-reading. By that I mean I read several books at once.
How I do this is by having different books in different locations. And also several books in the same location. I have a preference for printed books. PDFs of books are handy occasionally for specific searches, but I really cannot read page upon page on a screen. Kindle may be nice, but I have not yet jumped on that bandwagon.
This multi-book effect is partially due to laziness, as when I am in the lounge I do not want to have to walk back upstairs to my study if I get a sudden urge to read or fetch a book. So I always have a half read book in the lounge. (Currently in the lounge I am re-reading the Kanban book by Anderson).
In my study I naturally have bookshelves loaded with computers books, especially java, and always a couple on my desk. I tend not to read too long in this room (then why is it called a study?), but a quick few paragraph while the pc is rebooting etc a few times a day. The reference books however might be referred to when needed. (Current study books are Seven Languages in Seven Weeks by Tate and Continuous Delivery by Humble & Farley)
On my bed side table there may be a fiction book. I don't really want to think too much when trying to sleep.
At work I would also have a mini library as I also tend to evangelise and lend books to colleagues. So I usually have a half read book or two or three there as well. Again for quick glances while rebooting etc, but also for longer reads during lunch if I go by myself. (Just finished ReWork by Fried & Heinemeier Hansson and Linchpin by Godin)
When I am out-and-about e.g. queuing for the till in a shop, waiting for the bus or at lunch when I forgot a book, I use the Aldiko e-book reader on my Android phone. Being a sucker for offers at O'Reilly for epub formatted books I have quite a few on my phone. Aldiko is excellent, but the small phone format is in essence rubbish for reading books over time. It is good for quick 1-5 minutes reads but no longer. And useless for reference books as the overview of the page is difficult. (97 Things Every Programmer Should Know by Henney is good for 5 minute reads)
For longer commutes by train or plane I tend to bring an actual book. If I do this regularly I should probably invest in a Kindle. (Read bits of Specification by Example by Adzic on the plane recently)
I tend do a lot of quick reads. As a stop gap filler between other events. Maybe work interrupts, maybe it is some element of ADHD, or probably our 6 months old daughter. This means I constantly waste time remembering the context of where I am in the book.
Longer reading periods is usually a holiday pattern. This would then often involve more fiction books. I read the Night's Dawn trilogy[2][3] over the last few holidays.
Some books I might skim read chapters to get a gist of but I know I can revisit them if I need to know those sections in detail in the future. Some books I never finish. I would say a good third or more in library I have not finished or barely read. Some turned out just not interesting, some are just skim read while others are consequences of multi-book-reading and lower down the reading priority list.
So this multi-book-reading means I do read a lot of books. But it also means I do not finish enough books. Not very "kanban" or "one-piece-flow" :)
Tuesday, 17 January 2012
SOPA protest page and 503 redirect
Tomorrow, 18t January 2012, many websites will protest against the US SOPA and PIPA acts. For example Wikipedia will go blank in protest for 24 hours.
For those intending to blackout their own websites (or in the future intend to do something similar, e.g. a more practical "web site temporarily down message" while doing an upgrade etc, here are a few tips:
First of all Google recommend you do not simple just change your website front page to a blank page or similar. As this can have repercussions on your SEO, ie your search ranking. Read more about it in this post on Google+ by a Googler. They recommend a 503 error response instead. Which indicates the site is temporarily down.
So a simple change of index.html is not recommended. Nor is a simple redirect in the html's meta header, nor a plain http 302 redirect. All this can affect your ranking.
I recommend (if using Apache 2) to use mod_rewrite in this manner:
This uses Alias to another folder so that the same message can be used for several virtual hosts. It uses a custom ErrorDocument to display a human readable blackout page. And it uses RewriteRule to redirect all requests to the stop-sopa page (except for request for /stop-sopa so that you don't get an infinite loop).
If you are looking for a page to use as the blackout page, there is a nice github project page for just that. An example can be viewed here.
For those intending to blackout their own websites (or in the future intend to do something similar, e.g. a more practical "web site temporarily down message" while doing an upgrade etc, here are a few tips:
First of all Google recommend you do not simple just change your website front page to a blank page or similar. As this can have repercussions on your SEO, ie your search ranking. Read more about it in this post on Google+ by a Googler. They recommend a 503 error response instead. Which indicates the site is temporarily down.
So a simple change of index.html is not recommended. Nor is a simple redirect in the html's meta header, nor a plain http 302 redirect. All this can affect your ranking.
I recommend (if using Apache 2) to use mod_rewrite in this manner:
RewriteEngine on
RewriteCond %{ENV:REDIRECT_STATUS} !=503
Alias /stop-sopa /var/www/stop-sopa
ErrorDocument 503 /stop-sopa/index.html
RewriteRule !^s/stop-sopa$ /stop-sopa [L,R=503]
This uses Alias to another folder so that the same message can be used for several virtual hosts. It uses a custom ErrorDocument to display a human readable blackout page. And it uses RewriteRule to redirect all requests to the stop-sopa page (except for request for /stop-sopa so that you don't get an infinite loop).
If you are looking for a page to use as the blackout page, there is a nice github project page for just that. An example can be viewed here.
Saturday, 14 January 2012
Agile project tools for personal/open source projects
Been briefly assessing some online free tools for agile task planning for a few personal FOSS projects.
A physical task board is perhaps the suggestion from the agile purists, however not useful for me (nor my family :)).
At work I often have to use the awful Quality Center. It is good for planning functional testing, but not much else. The user interface is painful, and only works on windows with IE.
But most project I have been on eventually drop it for the more developer friendly Jira by Atlassian. Its UI gets cleaner and cleaner. And is great for Scrum projects since the intergration of GreenHopper. It is however very feature rich which is good and bad, and sometimes quite slow. I recommend Jira for distributed larger organisations. It is however an overkill for my needs.
I have been using Pivotal Tracker for some of my projects for a few years. It is a great tool. For scrum projects it is the tool I would recommend the most. They recently started charging but it is still free for public projects. It is however very iteration/scrum centric and as such not useful for my more Kanbanish time irrelevant requirements.
So I started to look at more tools (and revisit some previous ones).
My requirements are:
Not all requirements have to be met.
Here are my initial impressions:
Time iterative centric.
Looks nice. Clean interface.
No WIP limit.
No kanban queue.
Got Icebox feature
Got Feature-chore-bug classification.
Fibonacci estimates.
Unlimited free public projects.
Kanban style flow.
Looks nice. Clean interface.
Columns can be renamed.
Got WIP limit.
No icebox. Can rename backlog icebox and rename another column backlog.
No estimates
Only 1 project on the free price plan.
FOSS projects can apply for free usage.
Kanban style flow.
Looks nice. Clean interface.
Columns can be renamed.
Got WIP limit.
Got Icebox feature
Got Feature-chore-bug classification.
T-shirt estimates.
Only 1 project on the free price plan.
No FOSS free plan.
Kanban style flow.
Clean interface.
Little confusing UI.
Got Kanban queues.
Got WIP limit.
Got Icebox (the "backlog").
No estimates.
Unlimited projects.
All plans are free.
Permissions are strange. No member can edit and public can only view. Either member view and edit with no public access, or public(anonymous) can view and edit!!
Kanban style flow.
Seems very feature rich. Perhaps too many features.
UI a little cluttered.
Tasks seems too much like post-it notes.
Only 1 project on the free price plan.
No FOSS free plan.
Scrum focused.
Looks nice.
Feature rich.
UI a little confusing.
No Icebox.
No WIP limit.
No kanban queue.
Fibonacci and t-shirt estimates.
10 project on the free price plan.
No FOSS free plan.
Kanban style flow.
Tasks seems too much like post-it notes.
No Icebox.
Got WIP limit.
Only 1 project on the free price plan.
FOSS projects can apply for free usage.
I may update this in the future when I get more impressions of the ones I use and if I find other tools.
My recommendations depends, but currently they are:
A physical task board is perhaps the suggestion from the agile purists, however not useful for me (nor my family :)).
At work I often have to use the awful Quality Center. It is good for planning functional testing, but not much else. The user interface is painful, and only works on windows with IE.
But most project I have been on eventually drop it for the more developer friendly Jira by Atlassian. Its UI gets cleaner and cleaner. And is great for Scrum projects since the intergration of GreenHopper. It is however very feature rich which is good and bad, and sometimes quite slow. I recommend Jira for distributed larger organisations. It is however an overkill for my needs.
I have been using Pivotal Tracker for some of my projects for a few years. It is a great tool. For scrum projects it is the tool I would recommend the most. They recently started charging but it is still free for public projects. It is however very iteration/scrum centric and as such not useful for my more Kanbanish time irrelevant requirements.
So I started to look at more tools (and revisit some previous ones).
My requirements are:
- Free, as in beer or near enough. $9/month and similar is too much for personal projects unless heavily used.
- Agile task board simulation
- Not time iteration based
- Simple functional UI, but not ugly
- Icebox feature for storing tasks/ideas not yet ready for the backlog
- Pivotal like Feature, Chore and Bug classification
- Limiting WIP
- Kanban queues
- Simple T-shirt or fibonacci estimates
Not all requirements have to be met.
Here are my initial impressions:
Pivotal Tracker
Time iterative centric.
Looks nice. Clean interface.
No WIP limit.
No kanban queue.
Got Icebox feature
Got Feature-chore-bug classification.
Fibonacci estimates.
Unlimited free public projects.
AgileZen
Kanban style flow.
Looks nice. Clean interface.
Columns can be renamed.
Got WIP limit.
No icebox. Can rename backlog icebox and rename another column backlog.
No estimates
Only 1 project on the free price plan.
FOSS projects can apply for free usage.
Kanbanery
Kanban style flow.
Looks nice. Clean interface.
Columns can be renamed.
Got WIP limit.
Got Icebox feature
Got Feature-chore-bug classification.
T-shirt estimates.
Only 1 project on the free price plan.
No FOSS free plan.
Kanbanpad
Kanban style flow.
Clean interface.
Little confusing UI.
Got Kanban queues.
Got WIP limit.
Got Icebox (the "backlog").
No estimates.
Unlimited projects.
All plans are free.
Permissions are strange. No member can edit and public can only view. Either member view and edit with no public access, or public(anonymous) can view and edit!!
Leankit
Kanban style flow.
Seems very feature rich. Perhaps too many features.
UI a little cluttered.
Tasks seems too much like post-it notes.
Only 1 project on the free price plan.
No FOSS free plan.
ScrumDO
Scrum focused.
Looks nice.
Feature rich.
UI a little confusing.
No Icebox.
No WIP limit.
No kanban queue.
Fibonacci and t-shirt estimates.
10 project on the free price plan.
No FOSS free plan.
Flow
Kanban style flow.
Tasks seems too much like post-it notes.
No Icebox.
Got WIP limit.
Only 1 project on the free price plan.
FOSS projects can apply for free usage.
I may update this in the future when I get more impressions of the ones I use and if I find other tools.
My recommendations depends, but currently they are:
- For large commercial projects Jira offer features and reports. And can be installed inside your firewall.
- For Scrum projects Pivotal Tracker offers the most complete package.
- For Kanban projects, the it depends on your own requirements and taste, but my current favourites are Kanbanery and AgileZen. Kanbanpad's no restrictions on number of projects is also tempting
Wednesday, 11 January 2012
Play! 2.0 in IntelliJ IDEA
Play! Framework 1.x supported creating an IntelliJ IDEA project by the command:
However while Play! Framework 2.0 is in beta that command does not work**.
So how do you get your Play! 2.0 project to open in IntelliJ IDEA? There are few different work arounds. Especially regarding integrating sbt.
However I have a quick way. For this to work you need both Play! 2.0 and Play! 1.2.x installed.
Create Play! 2.0 project:
(I am assuming it was a Java project that you chose)
Rename project folder:
Create Play! 1.x project:
Create IntelliJ project:
Move IntelliJ files to Play! 2.0 project:
Remove 1.x project and rename 2.0 folder:
Now you can open IntelliJ and then go to file/open project, then find and open helloworld/helloworld.ipr.
There will be some issues such as libraries etc but this a good start. For further tips try these suggestions.
** As of 11th of January 2012 it is not present in Play! 2.0. I fully expect Play! to create an idealize, eclipsify, netbeansify etc as soon as 2.0 is stable.
play idealize.However while Play! Framework 2.0 is in beta that command does not work**.
So how do you get your Play! 2.0 project to open in IntelliJ IDEA? There are few different work arounds. Especially regarding integrating sbt.
However I have a quick way. For this to work you need both Play! 2.0 and Play! 1.2.x installed.
Create Play! 2.0 project:
/usr/local/lib/play-2.0-beta/play new helloworld(I am assuming it was a Java project that you chose)
Rename project folder:
mv helloworld helloworld2Create Play! 1.x project:
/usr/local/lib/play-1.2.4/play new helloworldCreate IntelliJ project:
cd helloworld;
/usr/local/lib/play-1.2.4/play idealizeMove IntelliJ files to Play! 2.0 project:
cd ..;
mv helloworld/helloworld.i* helloworld2/Remove 1.x project and rename 2.0 folder:
rm -rf helloworld;
mv helloworld2 helloworldNow you can open IntelliJ and then go to file/open project, then find and open helloworld/helloworld.ipr.
There will be some issues such as libraries etc but this a good start. For further tips try these suggestions.
** As of 11th of January 2012 it is not present in Play! 2.0. I fully expect Play! to create an idealize, eclipsify, netbeansify etc as soon as 2.0 is stable.
Wednesday, 16 November 2011
Do not rewrite
Etiketter:
agile,
architecture,
code,
legacy,
systems
Just don’t do it
Instead evolve
Many (99.99%) of developers continually insist that whichever application they work on needs a rewrite. As in scratch everything, redesign the architecture and technology choices and rewrite every component because the current choice and state is not ideal. I used to feel and say this as well.
Any code I have written (and even worse if written by others) which I need to update/maintain will look bad after only 6 months. After 2 years it smells of bad architecture design and logic. After 5 years the code base feels like a mess, totally unmaintainable.
This is slightly because technology has moved on, and my skills or rather preferences have evolved. But it is mostly due to that over time any application has been updated due to bug fixes and new features, and the maintainance might have been handed over to other people several times, with different skill level or understanding of the architecture. This bloats the code structure and exceeds what the original and probably quite tidy structure was intended for.
Any further maintenance is more and more costly as it takes longer and/or more people to make any changes to the application. And any innovation is muffled as inspiration is dampened.
Many companies makes a good living when other companies outsources the costly maintenance of these systems.
I used to feel the need to rewrite in every new assignment but not anymore. My advice now is: Please do not rewrite
Why are rewrites a bad idea?
No value
If you undertake a rewrite of your core application it will be very costly for your business, with no initial return on investment. Take 6 -12 months of pure losses and in the end the customer (internal or external) has not received any extra value. Most likely they will have less features as scope was reduced, but rarely any new features. Future performance gains and maintainability is a plus but not of value there and then. So it is a huge hit on fragile budgets or even liquidity of smaller companies, with little to no benefit.
This is the main reason not to rewrite, the business side gains nothing from it, it is a pure expense. Any reasonable business person would not approve it. And any future relationship will quite likely be very damaged.
Never finishing
Another risk is that the rewrite takes too long and either never really finishes or is cancelled. Making it an expensive task with no value now nor in the future.
Also rewrites that span a long time usually suffers from that the decisions made at the beginning start to look outdated and wrong before it is even in production and before any future value is gained.
However I do not mean you should not evolve your application. Do not keep status quo either. If it is an important core application, you should evolve it, but not perform a total rewrite.
Instead do it in smaller steps while adding value and features for the customer. This way your product is cleaner and up to date, but the business side does not take a huge hit. This is also much more predictable and less volatile. Which make is less likely the project will be cancelled or worse for smaller companies which could go bankrupt.
How do you go about evolving and not rewriting? This very much depends on your type of application.
(If it is a tiny application, go ahead rewrite it. Just make sure the dependant systems are aware of this).
Refactor
On a minor level, continual process of refactoring, with larger refactoring a frequent acceptable task. This should slow down code rot, postponing the need for larger changes a while.
Modularisation
Eventually your application will need a drastic change. If by chance your original design was quite modular or less coupled with other systems you made evolving smoother. If not, try to modularise your current design.
By being modular you can still perform rewrites, but on much smaller scale, taking one module at the time. So that it is not a 12 months loss, but perhaps 1 month loss. While normal work on other modules can still continue. Much less likely to kill your project/company.
Parallel systems
Instead of a hard one day switch over from the old system to the new system, instead run two systems in parallel for awhile, the old legacy system and the new clean system.
Skinny interface adapter layer
If modularised or not, a cleaner way to evolve and replace sections of your application is to introduce an adapter layer for your interface, and keeping it skinny.
This interface adapter only relays between your system/application and other systems. When other systems only talk via the adapter you are free to change your system without affecting external systems.
More importantly you can now split your system into two systems and relay certain calls to the new system instead of the old, without external systems are aware nor affected by this.
The adapter layer is not always possible, or people are resistant to introduce yet another layer/system, but it really makes your much more adaptable in changing your own architecture, you can move elements to a new system and roll back without external costs.
No logic in adapter layer
Do not ever put any logic in interface adapter. Otherwise you end up with yet another system to maintain. Unfortunately this happens quite often when bad planning and management leads to shortcuts which adds logic in the adapter layer. And then you have just multiplied your maintenance costs. And believe me that logic is then rarely temporary.
ESB
Do not interpret this adapter layer requirement as a reason for introducing an ESB. ESB tend to just camouflage the spaghetti integration code, and often introduce above mention unwanted logic. However if you already have an ESB in place it can perform the adapter layer requirement.
Legacy system becomes adapter layer
Another method implementing the adapter layer is to use the existing system to relay calls to the new system instead of handling them themselves. Internally in the existing legacy system you have to do this anyway on each element migrated to the new system, so you can also allow external system to do this.
It is however cleaner to have a separate adapter layer. This will be easier to maintain as probably designed on newer technology and platforms. And allows you switch off the legacy system when you can. But also makes so less likely for people to just decide to use the legacy system instead.
Topic / feature
An even better rewrite strategy/evolution than per module is per topic / feature. This is an even smaller grained change and less risky. As part of more agile strategies you can move one feature at the time to the new clean system.
With an adapter layer this switching is smooth, but not restricted to an adapter layer. Without the adapter you just have more administration of changing every dependant system for each feature moved.
Feature toggles might be part of this strategy.
New feature -> new system
Update feature -> new system
Every new feature request is naturally going to be implemented on the new system. But a good method of choosing which existing features to move is choosing a feature which there is a requirement to modify. Then not touch it on the old system but instead rewrite and update it on the new system. This is then a good carrot/stick way to ensure the migration is performed. And ensures that the company receive some value for each rewrite.
Do not update old(legacy) system/application
Another very important rule is not to update the old legacy system at all when each feature/module is migrated off it. It is so tempting to make a shortcut and update both system as perhaps you have not introduced an adapter layer or instructed enough external systems to use the new system. This will kill your migration/rewrite.
For this step the leadership of your team needs to be firm about and insist on. Do not end up maintaining two systems (or 3 with the adapter layer).
Ensure legacy system delegates
The legacy system will undoubtedly internally refer to the rewritten and migrate module or feature. You need to ensure the legacy system now delegates internally to the new system. This is the only update you should do on the old system. Otherwise again you will need to maintain two systems, and run the risk of the legacy system and the new system executes slightly differently for the same task, leading to people insisting on using the legacy system.
Kill the legacy system
You need to plan and ensure the old system is eventually switched off. Otherwise you will still need to keep people and skills on how to maintain that system. It may be tempting and erroneously to leave some elements behind that should be on the new architecture.
There may be some elements not needed on new system or unrelated that is left on the old system. But probably better to even move these to another new system than keep the old system lying around and increasing the cost of maintaining that small element as well.
Kill the adapter layer?
Once you killed the legacy system, do you want to keep the adapter layer? Keep in mind you might want to move on from the new system when it starts to rot as well. However it may be tidier and less complicated to kill the adapter layer as well. I would kill it and if needed in the future reintroduce it instead.
Do not stop half way
If for some reason the migration to a new system is stopped, either due to reprioritisation, lack of progress etc, then you are stuck with maintaining not 1 but 3 systems. Many companies end up burned by this and is mostly down to not strong enough leadership/management.
So my point is to rewrite small topics/features then modules, but never the whole application. This way value is introduced along the way without a budgetary black hole.
References
Many, but some from former colleague Anders Sveen
Instead evolve
Many (99.99%) of developers continually insist that whichever application they work on needs a rewrite. As in scratch everything, redesign the architecture and technology choices and rewrite every component because the current choice and state is not ideal. I used to feel and say this as well.
Any code I have written (and even worse if written by others) which I need to update/maintain will look bad after only 6 months. After 2 years it smells of bad architecture design and logic. After 5 years the code base feels like a mess, totally unmaintainable.
This is slightly because technology has moved on, and my skills or rather preferences have evolved. But it is mostly due to that over time any application has been updated due to bug fixes and new features, and the maintainance might have been handed over to other people several times, with different skill level or understanding of the architecture. This bloats the code structure and exceeds what the original and probably quite tidy structure was intended for.
Any further maintenance is more and more costly as it takes longer and/or more people to make any changes to the application. And any innovation is muffled as inspiration is dampened.
Many companies makes a good living when other companies outsources the costly maintenance of these systems.
I used to feel the need to rewrite in every new assignment but not anymore. My advice now is: Please do not rewrite
Why are rewrites a bad idea?
No value
If you undertake a rewrite of your core application it will be very costly for your business, with no initial return on investment. Take 6 -12 months of pure losses and in the end the customer (internal or external) has not received any extra value. Most likely they will have less features as scope was reduced, but rarely any new features. Future performance gains and maintainability is a plus but not of value there and then. So it is a huge hit on fragile budgets or even liquidity of smaller companies, with little to no benefit.
This is the main reason not to rewrite, the business side gains nothing from it, it is a pure expense. Any reasonable business person would not approve it. And any future relationship will quite likely be very damaged.
Never finishing
Another risk is that the rewrite takes too long and either never really finishes or is cancelled. Making it an expensive task with no value now nor in the future.
Also rewrites that span a long time usually suffers from that the decisions made at the beginning start to look outdated and wrong before it is even in production and before any future value is gained.
Evolve
However I do not mean you should not evolve your application. Do not keep status quo either. If it is an important core application, you should evolve it, but not perform a total rewrite.
Instead do it in smaller steps while adding value and features for the customer. This way your product is cleaner and up to date, but the business side does not take a huge hit. This is also much more predictable and less volatile. Which make is less likely the project will be cancelled or worse for smaller companies which could go bankrupt.
How do you go about evolving and not rewriting? This very much depends on your type of application.
(If it is a tiny application, go ahead rewrite it. Just make sure the dependant systems are aware of this).
Refactor
On a minor level, continual process of refactoring, with larger refactoring a frequent acceptable task. This should slow down code rot, postponing the need for larger changes a while.
Modularisation
Eventually your application will need a drastic change. If by chance your original design was quite modular or less coupled with other systems you made evolving smoother. If not, try to modularise your current design.
By being modular you can still perform rewrites, but on much smaller scale, taking one module at the time. So that it is not a 12 months loss, but perhaps 1 month loss. While normal work on other modules can still continue. Much less likely to kill your project/company.
Parallel systems
Instead of a hard one day switch over from the old system to the new system, instead run two systems in parallel for awhile, the old legacy system and the new clean system.
Skinny interface adapter layer
If modularised or not, a cleaner way to evolve and replace sections of your application is to introduce an adapter layer for your interface, and keeping it skinny.
This interface adapter only relays between your system/application and other systems. When other systems only talk via the adapter you are free to change your system without affecting external systems.
More importantly you can now split your system into two systems and relay certain calls to the new system instead of the old, without external systems are aware nor affected by this.
The adapter layer is not always possible, or people are resistant to introduce yet another layer/system, but it really makes your much more adaptable in changing your own architecture, you can move elements to a new system and roll back without external costs.
No logic in adapter layer
Do not ever put any logic in interface adapter. Otherwise you end up with yet another system to maintain. Unfortunately this happens quite often when bad planning and management leads to shortcuts which adds logic in the adapter layer. And then you have just multiplied your maintenance costs. And believe me that logic is then rarely temporary.
ESB
Do not interpret this adapter layer requirement as a reason for introducing an ESB. ESB tend to just camouflage the spaghetti integration code, and often introduce above mention unwanted logic. However if you already have an ESB in place it can perform the adapter layer requirement.
Legacy system becomes adapter layer
Another method implementing the adapter layer is to use the existing system to relay calls to the new system instead of handling them themselves. Internally in the existing legacy system you have to do this anyway on each element migrated to the new system, so you can also allow external system to do this.
It is however cleaner to have a separate adapter layer. This will be easier to maintain as probably designed on newer technology and platforms. And allows you switch off the legacy system when you can. But also makes so less likely for people to just decide to use the legacy system instead.
Topic / feature
An even better rewrite strategy/evolution than per module is per topic / feature. This is an even smaller grained change and less risky. As part of more agile strategies you can move one feature at the time to the new clean system.
With an adapter layer this switching is smooth, but not restricted to an adapter layer. Without the adapter you just have more administration of changing every dependant system for each feature moved.
Feature toggles might be part of this strategy.
New feature -> new system
Update feature -> new system
Every new feature request is naturally going to be implemented on the new system. But a good method of choosing which existing features to move is choosing a feature which there is a requirement to modify. Then not touch it on the old system but instead rewrite and update it on the new system. This is then a good carrot/stick way to ensure the migration is performed. And ensures that the company receive some value for each rewrite.
Do not update old(legacy) system/application
Another very important rule is not to update the old legacy system at all when each feature/module is migrated off it. It is so tempting to make a shortcut and update both system as perhaps you have not introduced an adapter layer or instructed enough external systems to use the new system. This will kill your migration/rewrite.
For this step the leadership of your team needs to be firm about and insist on. Do not end up maintaining two systems (or 3 with the adapter layer).
Ensure legacy system delegates
The legacy system will undoubtedly internally refer to the rewritten and migrate module or feature. You need to ensure the legacy system now delegates internally to the new system. This is the only update you should do on the old system. Otherwise again you will need to maintain two systems, and run the risk of the legacy system and the new system executes slightly differently for the same task, leading to people insisting on using the legacy system.
Kill the legacy system
You need to plan and ensure the old system is eventually switched off. Otherwise you will still need to keep people and skills on how to maintain that system. It may be tempting and erroneously to leave some elements behind that should be on the new architecture.
There may be some elements not needed on new system or unrelated that is left on the old system. But probably better to even move these to another new system than keep the old system lying around and increasing the cost of maintaining that small element as well.
Kill the adapter layer?
Once you killed the legacy system, do you want to keep the adapter layer? Keep in mind you might want to move on from the new system when it starts to rot as well. However it may be tidier and less complicated to kill the adapter layer as well. I would kill it and if needed in the future reintroduce it instead.
Do not stop half way
If for some reason the migration to a new system is stopped, either due to reprioritisation, lack of progress etc, then you are stuck with maintaining not 1 but 3 systems. Many companies end up burned by this and is mostly down to not strong enough leadership/management.
So my point is to rewrite small topics/features then modules, but never the whole application. This way value is introduced along the way without a budgetary black hole.
References
Many, but some from former colleague Anders Sveen
Tuesday, 30 August 2011
Null is okay
Most of the bugs that you find or are reported to you in java applications are NullPointerExceptions [1]. NPE are rife in the beginning of a product’s life cycle, and they never go away.
The consequences are bug fixing which checks if:
So this original pizza ordering method:
Morphs into this:
(Or alternativly many “assert pizzaOrder != null” that are useless in production)
Madness, and very messy.
Blinkered coding standard:
This happens when code ownership is not clear, when desire for clean code is not existant, when fixing bugs with blinkers on.
When developers are time constrained or scared of changing code.
When development is outsourced but authority and knowledge is not, so refactoring is avoided or not part of SLA.
Why are we so afraid of NullPointerExceptions?
Null is okay. Null is exactly that. Don't avoid it like the plague, but try to not pass the problem to others. Tidy up your gateway interfaces, trust internally null does not happen. If it does then don't hide the problem.
Do not check for null inside you application. Check for null on your edges:
Validate user interface inputs. “Users” are always causing trouble, but we can not live without customers. Validate the essentials, but remember duck walking, don’t scare your customers with perfection validation.
Validate external 3rd party api interfaces. Probably validate internal api interfaces. Remember duck walking. If an internal system quite close to your application erroneously passes null values, fix that system instead. It is a gray zone however so some null pointer validation is fine. Just don’t complicate your system because of the laziness of another within your team/company.
Do not validate calls between your applications layers. Trust the edges has validated the data. Use unit tests to ensure integrity.
If you encounter a Null or anything resembling empty when not possible etc: throw a NullPointerException. Don’t sugar coat and masquerade it as something else. An NPE is an NPE! It says what it is on the tin! It is easily understood by all developers, and easily traced. Obviously don’t show that to the end user, but don’t catch it too far down your stack.
If you get bug report about an NPE. Fix the edge validation, factory generation or inside the api call. Do not add a “== null“ condition inside your application.
It is easier to avoid internal NPE by not passing external DTO and domain objects through your own application. If you act on it, it should be your applications objects, not an external one. This avoids complicated null pointer validation.
“Clean Code” book suggest “specific case object” for method calls that may previously return null. I am not a big fan, but it may solve internal api NPEs. I definitely support the pattern of returning empty collections in find* calls instead of null when nothing was found.
How would I deal with Null in the Pizza Order scenario?
PizzaOrder object should be validated beforehand in the interface layer. It is not the responsibility of this method.
Same layer before this method should call a factory method that sets pizzas collection to an empty collection if no pizzas.
Kitchen is always injected or constructed externally to this method and not its responsibility.
Factories in mapping/creating the PizzaOrder before this method call should also ensure no Pizza objects are Null.
The one possible valid check is if there are any pizzas in the order. The
So Null is okay and NullPointerException can be your friend.
The consequences are bug fixing which checks if:
- input is null
- if interface calls return null
- if method calls return null
- if properties and collections are null
So this original pizza ordering method:
public void orderPizza(PizzaOrder pizzaOrder){
for( Pizza pizza : pizzaOrder.getPizzas()){
kitchen.makePizza(pizza);
}
}
Morphs into this:
public void orderPizza(PizzaOrder pizzaOrder){
if( pizzaOrder != null) {
if( pizzaOrder.getPizzas() != null ||
Collection.isEmpty(pizzaOrder.getPizzas())) {
if( kitchen != null) {
for( Pizza pizza : pizzaOrder.getPizzas()){
if( pizza != null) {
kitchen.makePizza(pizza);
} else {
throw PizzeriaTechnicalException(“No pizza!!!”);
}
}
} else {
throw PizzeriaTechnicalException(“No kitchen!!!”);
}
} else {
throw PizzeriaTechnicalException(“No pizzas in order!!!”);
}
} else {
throw PizzeriaTechnicalException(“No pizza order!!!”);
}
}
(Or alternativly many “assert pizzaOrder != null” that are useless in production)
Madness, and very messy.
Blinkered coding standard:
This happens when code ownership is not clear, when desire for clean code is not existant, when fixing bugs with blinkers on.
When developers are time constrained or scared of changing code.
When development is outsourced but authority and knowledge is not, so refactoring is avoided or not part of SLA.
Why are we so afraid of NullPointerExceptions?
What is wrong with null?
Null is okay. Null is exactly that. Don't avoid it like the plague, but try to not pass the problem to others. Tidy up your gateway interfaces, trust internally null does not happen. If it does then don't hide the problem.
Do not check for null inside you application. Check for null on your edges:
Validate user interface inputs. “Users” are always causing trouble, but we can not live without customers. Validate the essentials, but remember duck walking, don’t scare your customers with perfection validation.
Validate external 3rd party api interfaces. Probably validate internal api interfaces. Remember duck walking. If an internal system quite close to your application erroneously passes null values, fix that system instead. It is a gray zone however so some null pointer validation is fine. Just don’t complicate your system because of the laziness of another within your team/company.
Do not validate calls between your applications layers. Trust the edges has validated the data. Use unit tests to ensure integrity.
Dealing with NPE/Null?
If you encounter a Null or anything resembling empty when not possible etc: throw a NullPointerException. Don’t sugar coat and masquerade it as something else. An NPE is an NPE! It says what it is on the tin! It is easily understood by all developers, and easily traced. Obviously don’t show that to the end user, but don’t catch it too far down your stack.
If you get bug report about an NPE. Fix the edge validation, factory generation or inside the api call. Do not add a “== null“ condition inside your application.
It is easier to avoid internal NPE by not passing external DTO and domain objects through your own application. If you act on it, it should be your applications objects, not an external one. This avoids complicated null pointer validation.
“Clean Code” book suggest “specific case object” for method calls that may previously return null. I am not a big fan, but it may solve internal api NPEs. I definitely support the pattern of returning empty collections in find* calls instead of null when nothing was found.
How would I deal with Null in the Pizza Order scenario?
PizzaOrder object should be validated beforehand in the interface layer. It is not the responsibility of this method.
Same layer before this method should call a factory method that sets pizzas collection to an empty collection if no pizzas.
Kitchen is always injected or constructed externally to this method and not its responsibility.
Factories in mapping/creating the PizzaOrder before this method call should also ensure no Pizza objects are Null.
The one possible valid check is if there are any pizzas in the order. The
Collection.isEmpty(pizzaOrder.getPizzas()) But it is not a null pointer check, as previous interfaces/factories will ensure a collection object exists. However the interface or application before this method call should probably have validated the actual order, to ensure an order contains pizzas!
So Null is okay and NullPointerException can be your friend.
Tuesday, 16 August 2011
Boobs, breast and Tits. Functional or attractive?
A different post than my normal rambling:
Our beautiful baby daughter was recently born, and is in the middle of breastfeeding. And this constant feeding got me thinking of how I was reacting to this exposure.
Basically at the moment I am only thinking of my partners boobs as a utility to feed our daughter. They have purely a functional property.
It is quite the same when I see other mothers feed their babies on park benches etc. Mostly I think it is just sweet or indifferent to it.
So basically this body part which often is connected with attraction and excitement, is by introducing a baby changing my perception completely. And I would guess this reaction is similar with most other males, at various degrees (and probably some exceptions).
But remove the baby and any pregnancy references, if I catch a glimpse of a breast or cleavage I still snigger like a teenager...
Well at least I got to have "boobs" in a blog title, instead of my normal geeky techy posts :)
Our beautiful baby daughter was recently born, and is in the middle of breastfeeding. And this constant feeding got me thinking of how I was reacting to this exposure.
Basically at the moment I am only thinking of my partners boobs as a utility to feed our daughter. They have purely a functional property.
It is quite the same when I see other mothers feed their babies on park benches etc. Mostly I think it is just sweet or indifferent to it.
So basically this body part which often is connected with attraction and excitement, is by introducing a baby changing my perception completely. And I would guess this reaction is similar with most other males, at various degrees (and probably some exceptions).
But remove the baby and any pregnancy references, if I catch a glimpse of a breast or cleavage I still snigger like a teenager...
Well at least I got to have "boobs" in a blog title, instead of my normal geeky techy posts :)
Tuesday, 3 May 2011
Ubuntu releases
Ps. This is not directly regarding the latest 11.04 Natty release with the infamous Unity UI. (As of 03.05.2011) I am on 10.04 Lucid on my desktop and 10.10 Maverick on my servers (I know, it should be the other way round...), so I have not even tried Unity yet.
Every 2 years one of these releases is a LTS, Long Term Support, version that they will support for 3 to 5 years. This is the release to use for more production systems.
Most software versions are stable enough, but however new divergences are often not polished enough and lack enough extensions, documentation, etc. E.g. Unity, Gnome 3, GDM2, Plymouth etc. For the initiated that will research solutions and like dabbling with new software this is not a big issue. For the vast majority of users that just wants something that works, this is risky and often backfires on their impression of Linux and Ubuntu.
Also a 6 months release schedule does mean quite a frequent upgrade requirement. Especially for the uninitiated non heavy Linux fanboys (if such a name exists). But you have to set the line somewhere, and a more frequent upgrade does mean a smaller delta difference and less chance of broken upgrades. On production servers however, 6 months upgrade schedule is a non starter. The LTS schedule is thus more suitable.
Also when an LTS version is released it is "promoted" as LTS immediately. Being a "major" release a lot of people upgrade to it soon after the release date, but there are still many teething errors before it is more "solid", usually then they release a X.x.1 version.
They should not apply the LTS "brand" to the version until the X.x.1 version a few months after the general X.x release. That way teething errors are never found in an LTS version, and more trust can be applied to the version branding.
They should include more backports to the LTS versions. After about a year my LTSes are a nuisance as their packages are too out of date.
Have a "major", "minor", "tiny" or better "solid", "stable", "unstable" release versioning? Or an unstable release every 4 months(fanboys), stable every 8 (desktops) and solid(servers)(LTS) every 16 months? Too much admin or release confusion perhaps?
While debian's stable/testing/unstable naming is a close match, Ubuntu has always been relible. But lately I have been reluctant to upgrade, waiting months before all initial problems are out of the way before I dabble. And now I usually skip a release or two every time.
And optionally change the schedule to more frequent LTS, or even 3 levels of stability/support releases.
Current release schedule
Ubuntu releases a new version every 6 months, in April and October. They are supported for 18 months. I do not really have an issue with this frequency.Every 2 years one of these releases is a LTS, Long Term Support, version that they will support for 3 to 5 years. This is the release to use for more production systems.
In practice
In practice this 6 month release schedule means a lot of bleeding edge software and versions go into every version. This is good. It means the distribution is up to date, and the software is pushed and cannot rest on its laurels and falter.Most software versions are stable enough, but however new divergences are often not polished enough and lack enough extensions, documentation, etc. E.g. Unity, Gnome 3, GDM2, Plymouth etc. For the initiated that will research solutions and like dabbling with new software this is not a big issue. For the vast majority of users that just wants something that works, this is risky and often backfires on their impression of Linux and Ubuntu.
Also a 6 months release schedule does mean quite a frequent upgrade requirement. Especially for the uninitiated non heavy Linux fanboys (if such a name exists). But you have to set the line somewhere, and a more frequent upgrade does mean a smaller delta difference and less chance of broken upgrades. On production servers however, 6 months upgrade schedule is a non starter. The LTS schedule is thus more suitable.
LTS
Unfortunately the LTS version is not always the version Ubuntu "promote". After a newer minor version has been released it takes a back burner. Too much emphasis I feel on promoting, discussing and supporting the newer versions and not backporting enough to the "stable" LTS version.Also when an LTS version is released it is "promoted" as LTS immediately. Being a "major" release a lot of people upgrade to it soon after the release date, but there are still many teething errors before it is more "solid", usually then they release a X.x.1 version.
Small modification
They should promote LTS to everyone, and let fanboys use the latest non LTS versions. This way the keen users will still be up to date, and will ensure fixes and velocity of the features. But they will keep a very stable version for the majority users.They should not apply the LTS "brand" to the version until the X.x.1 version a few months after the general X.x release. That way teething errors are never found in an LTS version, and more trust can be applied to the version branding.
They should include more backports to the LTS versions. After about a year my LTSes are a nuisance as their packages are too out of date.
Large modification
LTS current biennial release schedule is a large gap. What about a LTS every year or 18 months?Have a "major", "minor", "tiny" or better "solid", "stable", "unstable" release versioning? Or an unstable release every 4 months(fanboys), stable every 8 (desktops) and solid(servers)(LTS) every 16 months? Too much admin or release confusion perhaps?
While debian's stable/testing/unstable naming is a close match, Ubuntu has always been relible. But lately I have been reluctant to upgrade, waiting months before all initial problems are out of the way before I dabble. And now I usually skip a release or two every time.
Summary
Basically change the promotion of LTS. Let the LTS be the default version. And do not call it LTS until the Ubuntu x.x.1 teething problem bugfix version is released.And optionally change the schedule to more frequent LTS, or even 3 levels of stability/support releases.
Wednesday, 20 April 2011
Required reading
Etiketter:
books,
development
Should a development team/department have a required reading list?
A list containing books (articles etc) that each member should read.
I think so.
Also “Should read” does not mean all books have to have been read before joining the team, more a list of books to read while in the team.
Some people are very productive and valuable without updating themselves. It is rare and uncommon however.
Some people though can read fast others not, some have no life and can read all weekend, others have 15 kids and no spare time. As long as they “are” reading I would be happy. 1 book per year or 10, it is all good.
To list suggestions for teams similar to mine (enterprise Java, finance sector):
Any comments or suggestions are welcom.
A list containing books (articles etc) that each member should read.
I think so.
Required
I am not sure if “required” is the right concept, “strongly encouraged”, incentive linked or similar is perhaps enough emphasis.Also “Should read” does not mean all books have to have been read before joining the team, more a list of books to read while in the team.
Why
To gather a common basis for discussions and choices within the team. To grow the competence of the team and ensure more correct decisions by having more relevant information. By reading and updating themselves the team is exposed to new ideas and can discuss and grow their understanding of this at work.Unanimous agreement
It should not be requirement to agree with the contents and aim of every book, however it should be a requirement to have read it so that it can be discussed. In favour or quite the opposite only adds to the discussion quality.Never read them all
You should never be finished with the the “Reading list”. If the list is too short and you have read all of them (or even based it on your own previously read books) then you become complacent and not open to further competences and ideas.List evolution
The list should grow continuously and irrelevant/outdated items should be pruned from the list. The list should perhaps be put into sections, and also prioritised if large.No reading
Team members that does not read nor update themselves in other ways, do you want them in your team? They may be great right now, but will they stay that way? I am always wary of people that do no expose themselves to new thinking. Can I really trust their convictions on a solution for an issue is the most prudent solution?Some people are very productive and valuable without updating themselves. It is rare and uncommon however.
Pretend readers
Some colleagues will perhaps say they have read or reading books from the list, but have no intention of reading anything. There should probably not be any need to quiz them to ensure knowledge, in due time their evaporating competences will show their true colours anyway.Book qualification
How do you select books on the list? Initially perhaps senior team members, architects etc could select a small selection of relevant books. But as the team grows the list should probably be suggested and voted on by the whole team to ensure common consensus and introduce newer not mainstream suggestions.Reading speed
An interesting issue would be how to ensure that people do actively read. That is not just up to the individual, but should perhaps be part of incentives, part of project cost and time allocations. That is a harder sell, but will pay itself over time.Some people though can read fast others not, some have no life and can read all weekend, others have 15 kids and no spare time. As long as they “are” reading I would be happy. 1 book per year or 10, it is all good.
Not related titles
Perhaps the list could contain subjects not related to the specific teams function. Fictional, philosophical books? Perhaps not.My reading list
If I were to list books I would probably include many items I have not read, just skimmed or even agree with. I also do not have a great amount of time to read books either, but I force myself. I get the odd for free for reviewing, buy a few related to projects and a few hobby subjects, totaling 4-8 per year.To list suggestions for teams similar to mine (enterprise Java, finance sector):
- Design patterns (Gamma, Helm, Johnson, Vlissides)
- Patterns of Enterprise Application Architecture (Fowler)
- Enterprise Integration Patterns (Hohpe, Woolf)
- Mythical Man Month (Brooks)
- Peopleware (DeMarco, Lister)
- Agile Project Management With Scrum (Schwaber)
- Succeeding with Agile (Cohn)
- Kanban (Anderson)
- Kanban and Scrum (Kniberg)
- Clean Code (Martin)
- Working Effectively with Legacy Code (Feathers)
- Effective Java (Bloch)
- Java Concurrency in Practice (Goetz)
- ReWork (Fried, Heinemeier Hansson)
- 97 Things Every Programmer Should Know (Henney)
- REST in Practice (Webber, Parastatidis, Robinson)
- Seven Languages in Seven Weeks (Tate)
- UML 2 for Dummies (Chonoles)
Any comments or suggestions are welcom.
Monday, 7 March 2011
Remove deadlines, increase productivity
In a recent project the tasks were always delegated and pointless deadlines set on everything. This seems so counter productive and ineffective compared to my past 5 years of agile based projects (inc maintenance).
As for task delegation I will need to evangelise more about the benefits of self organising and ownership through bottom up delegation, WIPs, velocity projection, prioritisation benefits etc.
My current beef is about noisy deadlines. Deadlines are for product owner / project managers. The team should never need to deal with deadlines. Especially when not meeting them has no consequence, so why have them at all? The team deals with prioritised tasks and if needed time boxed tasks. The tasks are divided up so that they are small enough so that a slip of 300% from estimation has no really impact as you are only talking about hours or days. However if the project velocity overt time does not seem to meet external deadlines then the scope of the backlog need to change, not to push developers/testers harder.
In agile teams they should not confuse deadlines for an estimation of how much work is left. If your current task may seem like another 2 days worth of work, do not set a deadline in two days. You may have interfering meetings or get stuck on a small issue that extend that tasks into next week. (Or hit an coding-happy-zone or eureka moment and finish in a few hours). Setting a deadline do not make you finish earlier and pushing developers/testers harder does not improve code quality. A task based deadline is only noise and stress.
Product owners can plan, communicate, etc with dates to customers, company boards or steering groups etc but they should be soft, movable dates and based on current project velocity, and no minor deadlines. If the velocity slows down, then reduce the scope of the backlog, or move the dates. Do not put pressure on the team with more micromanaged deadlines. It will only be noise and slow down development. The team does not really need to know specific dates apart perhaps from general roadmap (lavalamp?)for big releases.
True, some dates can not be moved, such as Christmas promotions, larger entities' deadlines etc, but with enough balls you would be surprised how many customer, senior management deadlines can be moved with good communication, even completely removed. Good project transparency and communication, and thus project velocity visibility is usually a much bigger relation and delivery benefit.
I agree that some pressure of delivery should exist but no cutthroat stress inducing deadlines. An agile process of pride in delivering tasks may quickly vanquish need for deadlines.
As for task delegation I will need to evangelise more about the benefits of self organising and ownership through bottom up delegation, WIPs, velocity projection, prioritisation benefits etc.
My current beef is about noisy deadlines. Deadlines are for product owner / project managers. The team should never need to deal with deadlines. Especially when not meeting them has no consequence, so why have them at all? The team deals with prioritised tasks and if needed time boxed tasks. The tasks are divided up so that they are small enough so that a slip of 300% from estimation has no really impact as you are only talking about hours or days. However if the project velocity overt time does not seem to meet external deadlines then the scope of the backlog need to change, not to push developers/testers harder.
In agile teams they should not confuse deadlines for an estimation of how much work is left. If your current task may seem like another 2 days worth of work, do not set a deadline in two days. You may have interfering meetings or get stuck on a small issue that extend that tasks into next week. (Or hit an coding-happy-zone or eureka moment and finish in a few hours). Setting a deadline do not make you finish earlier and pushing developers/testers harder does not improve code quality. A task based deadline is only noise and stress.
Product owners can plan, communicate, etc with dates to customers, company boards or steering groups etc but they should be soft, movable dates and based on current project velocity, and no minor deadlines. If the velocity slows down, then reduce the scope of the backlog, or move the dates. Do not put pressure on the team with more micromanaged deadlines. It will only be noise and slow down development. The team does not really need to know specific dates apart perhaps from general roadmap (lavalamp?)for big releases.
True, some dates can not be moved, such as Christmas promotions, larger entities' deadlines etc, but with enough balls you would be surprised how many customer, senior management deadlines can be moved with good communication, even completely removed. Good project transparency and communication, and thus project velocity visibility is usually a much bigger relation and delivery benefit.
I agree that some pressure of delivery should exist but no cutthroat stress inducing deadlines. An agile process of pride in delivering tasks may quickly vanquish need for deadlines.
- agile-commentary.blogspot.com/2009/06/are-deadlines-important.html
- developers.slashdot.org/story/06/09/28/1242257/Good-Agile-mdash-Development-Without-Deadlines
- steve-yegge.blogspot.com/2006/09/good-agile-bad-agile_27.html
- www.slideshare.net/jurgenappelo/agile-management-authority-delegation
- www.noop.nl/2009/10/why-we-delegate-the-darkness-principle.html
- www.informit.com/articles/article.aspx?p=1675546&seqNum=6
- blog.adsdevshop.com/2010/02/09/task-delegation-has-no-place-in-agile/
- stackoverflow.com/questions/63100/scrum-process-management-tips-pitfalls-ideas
Wednesday, 26 January 2011
Create, populate and reset a dynamic database (HSQLDB,Hibernate,SQLMaven,DbUnit)
Thought I'd jot down how I create, populate and reset databases for development and testing. I use this method in my Java based pet projects such as Snaps [app][code] and Wishlist [app][code]. I also include this setup as default in my project template.
My projects are Maven based, using Jetty as java container through its maven plugin. In addition I also use JRebel, IntelliJ IDEA and Ubuntu, a setup I described in this howto: Ubuntu + IntelliJ + Maven + Jetty + JRebel, but none of these are required.
This setup is enables a fluid, very dynamic and quick development process. In addition I then use the in memory database HSQLDB for quick and standalone database interaction.
Using JPA/Hibernate with hsqldb, my tables can be created automatically when I start Jetty. However in developement and testing I prefer to have some default data pre populated.
This is where DbUnit comes in to play. With DBunit's maven plugin I can export my current data and populate future databases with the same data. As it is all easy to read XML I can edit manually the basic stub data as well.
However DbUnit can not create the database as it is run before jetty:run action, so I use the SQL-Maven plugin for this purpose. Again SQL Maven needs to know what tables to create, so I use the Maven Hibernate3 Plugin to export a schema from the JPA annotations.
This combination allows me to:
To export the JPA annotations into a database schema we include the Maven Hibernate3 Plugin in your pom.xml. This is quite a long plugin section as it defines a few dependencies which otherwise may be overriden by the plugin and lead to problems like this mapping exception.
To run the schema creation action on its own:
Note: We need to compile the JPA classes so that the plugin can extract annotation information from them.
Note2: that Hibernate/Red Hat have a bad track record of including their plugins and releases properly in maven central repository, so you might either want to adjust your dependeinces and repositories accordingly. Or you can use my proxy repository:
Please if your project is popular do not use my repo as it will explode my ec2/S3 data bandwidth usage!
To create your database from this schema with SQL-Maven, add this to you pom.xml
To run the database creation action on its own:
To populate your database via DbUnit, add this to you pom.xml
First you need data to export. I suggest you run jetty:run action and add/use the application to create data in the database. Eg.: Register a user, create default domain objects, or whatever your application uses.
You then stop the application and export the data created:
These are by default exported to target/dbunit/export.xml.
These are a good base data. You may try and adjust and extend this file if you like, but be aware that you might brake it, so prehaps try unadjusted initially.
Copy the export file to test/data so that it can be part of your source control etc. You may have noticed I already included the test/data as a src element in the plugin.
Run the database population action (remember to have a new clean database):
Example configuration of all these plugins can be found in this project's pom.xml.
Above are the basic steps. These I then aggregate into a couple of common one liners:
Create database:
Create and populate database:
Reset database:
Clean, rebuild database and run jetty:
As you can see these one liners can be quite long so I again wrap these into bash scripts that I put in a bin folder.
You can extend this further to include these plugins as enabled by default and the maven goals as part of your other goals: E.g. include hbm:export and sqlmaven as part of install or test command or dbunit:operation with the jetty:run command, so that they are totally automatic. However I prefer a bit more control of when my data is reset etc.
With these plugins I can in one command create and populate my database, so that my app is up and running with data very quickly. Hope this is of use to others.
My projects are Maven based, using Jetty as java container through its maven plugin. In addition I also use JRebel, IntelliJ IDEA and Ubuntu, a setup I described in this howto: Ubuntu + IntelliJ + Maven + Jetty + JRebel, but none of these are required.
This setup is enables a fluid, very dynamic and quick development process. In addition I then use the in memory database HSQLDB for quick and standalone database interaction.
Using JPA/Hibernate with hsqldb, my tables can be created automatically when I start Jetty. However in developement and testing I prefer to have some default data pre populated.
This is where DbUnit comes in to play. With DBunit's maven plugin I can export my current data and populate future databases with the same data. As it is all easy to read XML I can edit manually the basic stub data as well.
However DbUnit can not create the database as it is run before jetty:run action, so I use the SQL-Maven plugin for this purpose. Again SQL Maven needs to know what tables to create, so I use the Maven Hibernate3 Plugin to export a schema from the JPA annotations.
This combination allows me to:
- Check out my project anywhere
- Create and populate the database with stub data with one command.
- Run the application via jetty
- Test the application straight away
- or develop dynamically with instant feedback
How to set up Hibernate, SQLMaven and DbUnit plugins
Prerequisites/assumptions
- Java
- Maven
- Jetty (can be tomcat maven plugin)
- JPA annotations
- HSQLDB (using file based hsqldb persistance to survive restarts)
JPA annotations -> Schema: Hibernate
To export the JPA annotations into a database schema we include the Maven Hibernate3 Plugin in your pom.xml. This is quite a long plugin section as it defines a few dependencies which otherwise may be overriden by the plugin and lead to problems like this mapping exception.
<profile>
<id>hbm-export</id>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>hibernate3-maven-plugin</artifactId>
<version>2.2</version>
<executions>
<execution>
<phase>process-classes</phase>
<goals>
<goal>hbm2ddl</goal>
</goals>
</execution>
</executions>
<configuration>
<components>
<component>
<name>hbm2ddl</name>
<implementation>jpaconfiguration</implementation>
</component>
</components>
<componentProperties>
<persistenceunit>${project.artifactId}</persistenceunit>
<outputfilename>schema.ddl</outputfilename>
<drop>false</drop>
<create>true</create>
<export>false</export>
<format>true</format>
</componentProperties>
</configuration>
<dependencies>
<dependency>
<groupId>hsqldb</groupId>
<artifactId>hsqldb</artifactId>
<version>${hsqldb.version}</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-entitymanager</artifactId>
<version>${hibernate.version}</version>
<exclusions>
<exclusion>
<groupId>cglib</groupId>
<artifactId>cglib</artifactId>
</exclusion>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-core</artifactId>
<version>${hibernate.version}</version>
<exclusions>
<exclusion>
<groupId>cglib</groupId>
<artifactId>cglib</artifactId>
</exclusion>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-annotations</artifactId>
<version>${hibernate.version}</version>
<exclusions>
<exclusion>
<groupId>cglib</groupId>
<artifactId>cglib</artifactId>
</exclusion>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
</profile>
To run the schema creation action on its own:
mvn -DskipTests \
-Dhibernate.dialect=org.hibernate.dialect.HSQLDialect \
-P hbm-export compile hibernate3:hbm2ddl;Note: We need to compile the JPA classes so that the plugin can extract annotation information from them.
Note2: that Hibernate/Red Hat have a bad track record of including their plugins and releases properly in maven central repository, so you might either want to adjust your dependeinces and repositories accordingly. Or you can use my proxy repository:
<repositories>
<repository>
<id>code-flurdy-repo</id>
<name>code@flurdy repository</name>
<url>http://code.flurdy.com/nexus/content/groups/noncentral</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>code-flurdy-repo</id>
<name>code@flurdy repository</name>
<url>http://code.flurdy.com/nexus/content/groups/noncentral</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>false</enabled></snapshots>
</pluginRepository>
</pluginRepositories>
Please if your project is popular do not use my repo as it will explode my ec2/S3 data bandwidth usage!
Schema -> Database: SQLMaven
To create your database from this schema with SQL-Maven, add this to you pom.xml
<profile>
<id>sqlmaven</id>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>sql-maven-plugin</artifactId>
<version>1.3</version>
<dependencies>
<dependency>
<groupId>hsqldb</groupId>
<artifactId>hsqldb</artifactId>
<version>${hsqldb.version}</version>
</dependency>
</dependencies>
<configuration>
<driver>${db.driverClassName}</driver>
<url>${db.url}</url>
<username>${db.username}</username>
<password>${db.password}</password>
<autocommit>true</autocommit>
<srcFiles>
<srcFile>target/hibernate3/sql/schema.ddl</srcFile>
</srcFiles>
</configuration>
</plugin>
</plugins>
</build>
</profile>
To run the database creation action on its own:
mvn -DskipTests -P sqlmaven sql:execute;Database population: DbUnit
To populate your database via DbUnit, add this to you pom.xml
<profile>
<id>dbunit</id>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>dbunit-maven-plugin</artifactId>
<version>1.0-beta-3</version>
<dependencies>
<dependency>
<groupId>hsqldb</groupId>
<artifactId>hsqldb</artifactId>
<version>${hsqldb.version}</version>
</dependency>
</dependencies>
<configuration>
<dataTypeFactoryName>org.dbunit.ext.hsqldb.HsqldbDataTypeFactory</dataTypeFactoryName>
<url>jdbc:hsqldb:file:${project.basedir}/target/db/build;shutdown=true</url>
<driver>org.hsqldb.jdbcDriver</driver>
<username>sa</username>
<password></password>
<type>CLEAN_INSERT</type>
<src>src/test/data/stub.xml</src>
</configuration>
</plugin>
</plugins>
</build>
</profile>
First you need data to export. I suggest you run jetty:run action and add/use the application to create data in the database. Eg.: Register a user, create default domain objects, or whatever your application uses.
You then stop the application and export the data created:
mvn -DskipTests -P dbunit dbunit:export;These are by default exported to target/dbunit/export.xml.
These are a good base data. You may try and adjust and extend this file if you like, but be aware that you might brake it, so prehaps try unadjusted initially.
Copy the export file to test/data so that it can be part of your source control etc. You may have noticed I already included the test/data as a src element in the plugin.
cp target/dbunit/export.xml test/data/stub.xml;Run the database population action (remember to have a new clean database):
mvn -DskipTests -P dbunit dbunit:operation;Example configuration of all these plugins can be found in this project's pom.xml.
Combined
Above are the basic steps. These I then aggregate into a couple of common one liners:
Create database:
mvn -o -DskipTests \
-Dhibernate.dialect=org.hibernate.dialect.HSQLDialect \
-P hbm-export,sqlmaven \
compile hibernate3:hbm2ddl sql:execute;Create and populate database:
mvn -o -DskipTests \
-Dhibernate.dialect=org.hibernate.dialect.HSQLDialect \
-P hbm-export,sqlmaven,dbunit \
compile hibernate3:hbm2ddl \
sql:execute dbunit:operation;Reset database:
rm -rf target/db;
mvn -o -DskipTests \
-Dhibernate.dialect=org.hibernate.dialect.HSQLDialect \
-P hbm-export,sqlmaven,dbunit \
compile hibernate3:hbm2ddl \
sql:execute dbunit:operation; Clean, rebuild database and run jetty:
mvn -o -DskipTests \
-Dhibernate.dialect=org.hibernate.dialect.HSQLDialect \
-P hbm-export,sqlmaven,dbunit \
clean compile hibernate3:hbm2ddl \
sql:execute dbunit:operation jetty:run;As you can see these one liners can be quite long so I again wrap these into bash scripts that I put in a bin folder.
You can extend this further to include these plugins as enabled by default and the maven goals as part of your other goals: E.g. include hbm:export and sqlmaven as part of install or test command or dbunit:operation with the jetty:run command, so that they are totally automatic. However I prefer a bit more control of when my data is reset etc.
Summary
With these plugins I can in one command create and populate my database, so that my app is up and running with data very quickly. Hope this is of use to others.
Thursday, 6 January 2011
InfoQ Presentation Video Enlarger
I like Infoq, and especially their presentations.
These presentation page has 3 elements:
This layout works very well, and basically shows you what you need.
But...
The window is a bit small (320x240), especially on larger resolutions screens. Especially when the presenter also shows some code, which is usually shown in the video window.
So I wrote a little user script which enlarges this video window (480x320).
This script is only for the Google Chrome browser. A similar Firefox based GreaseMonkey script should be easy to convert.
To install click on my
InfoQ Presentation Video Enlarger user script available at
www.ivar.co.uk/cargo/infoq-enlarge-presentation.chrome.user.js.
You can download and look at the js in a text editor first.It really is a very small and easy to understand script.
Anyone can contribute changes if desired.
These presentation page has 3 elements:
- Top left: One small video showing the person presenting
- Top right: One small window describing the presentation and presenters
- Bottom: One large window showing the actual presentation slide
This layout works very well, and basically shows you what you need.
But...
The window is a bit small (320x240), especially on larger resolutions screens. Especially when the presenter also shows some code, which is usually shown in the video window.
So I wrote a little user script which enlarges this video window (480x320).
This script is only for the Google Chrome browser. A similar Firefox based GreaseMonkey script should be easy to convert.
To install click on my
InfoQ Presentation Video Enlarger user script available at
www.ivar.co.uk/cargo/infoq-enlarge-presentation.chrome.user.js.
You can download and look at the js in a text editor first.It really is a very small and easy to understand script.
Anyone can contribute changes if desired.
Subscribe to:
Posts (Atom)