2011 Tech Conferences Sum Up

In our eternal seek for knowledge, 2011 found us travelling to San Francisco, Portland and Sao Pablo to keep learning about technology. This is a late sum up of the conferences we attended.

JQueryCon (April - San Francisco)

A business trip turned into an educational one when we registered ourselves at the jquery conference. Two days of javascript, html5, jquery and all kind of frontend related stuff. Gabriel Baños and Mariano Cortesi went to this one. All the slides can be found here.
They are maybe a little old right now, but still, we recommend checking these talks:

• Mobile Performance by Steve Souders (slides)
• Progressive Enhancement 2.0: Because the Web isn't Print by Nicholas Sackas (slides)
• jQuery + Popcorn.js = Interactive, Immersive HTML5 Video Experiences by Rick Waldron (slides)
• Deferreds - Putting Laziness to Work by Dan Heberden (slides)
• Filling the HTML5 & CSS3 Gaps with Polyfills and Shims by Rey Bango (slides)

OSCon'11 (July - Portland)

Great technology conference organized by O'Reilly, with a focus on Open Source. Juan Codagnone, our open source advocate, attended. All decks & some presentation videos are already available here.

We recommend:

• What Would You Do With Your Own Google? by Steve Yegge (details and video)
• 4 Practical Uses for Domain Specific Languages by Neal Ford (details and slides)
• Java: The Good, Bad, and Ugly Parts by Josh Bloch. (details and video)
• The Expressiveness of Go by Rob Pike (details and slides)
• What Every Data Programmer Needs to Know About Disks by Ted Dziuba (details and slides)

QCon SP (September - Sao Pablo)

Second edition of qcon conference in Brazil. A Zauber crowd went to this event, our representatives were Juan Codagnone, Fernando Zunino, Christian Nardi and Marcelo Turrin. The conference was organized by our friends at Caelum. Most of the slides from the presentations are now available here.



We recommend:

• Performance Oriented Design by Rodrigo Albani de Campos (slides)
• Divida Tecnica by Alexandre Freire (slides)

LatAm Con on Agile Methodologies (October - Buenos Aires)

In our beloved city, an agile methodologies conference. Cecilia Hagge went there to represent Zauber. Lots of talks and workshops by local and international agile experts. Subjects as Story Writing, Estimation and Continuous Delivery where the stars. You can find a complete post on this conference on our blog.

RubyCon AR (November - Buenos Aires)

This one didn't require much travel, maybe a metro or bus ticket. Nevertheless, it was really interesting to make the first steps onto the ruby community in Argentina. Zauber has been for some time know mostly java centric, but this year we started learning new languages and platforms such as Scala, Groovy and, in this case Ruby. We hope to learn much more about it in 2012, we already have several ruby enthusiasts at the office. Juan Codagnone, Fernando Zunino and Juan Pablo Royo attended the conference.
We recommend:

• Optimizing for Happiness by Tom Preston-Werner (slides)
• Un Cuento de Tres Árboles by Scott Chacon (slides)
• Who Makes the Best Asado? by Aaron Patterson (slides)
• Monitoring with Syslog and Event Machine by Patrick Huesler (slides)
• Embrace NoSQL and Eventual Consistency with Ripple by Sean Cribbs (slides)

AnDevCon II (November - San Francisco)

Our first android conference. Mariano Cortesi and Andrés Moratti went to San Francisco to represent Argentina in this event. We really enjoyed getting to know the people in the android community. Some really interesting talks took place at the conference. The presentation decks are now available here.
Some talks we recommend:

• Deep Drive into Android Security by Aleksandar Gargenta (deck)
• Pragmatic Android Layouts by Kirill Grouchnikov (deck)
• RoboGuice by Michael Burton (deck1 , deck2)
• Tunning Android Applications (deck1, deck2)

QCon SF (November - San Franciso)

Final but not last, another big conference. Quite a classic in San Francisco, the QCon. A Conference organized by the guys at InfoQ, with a focus on architecture solutions, java, startups, and emerging technologies. Mariano Cortesi and Andrés Moratti represented Zauber at this conference. Videos for all the talks will appear an InfoQ along the next months, you can check the schedule here. If you can't wait, you can check the slides for all presentation at the qconsf site.




Some talks we recommend:

• Software Naturalism - Embracing the Real Behind the Ideal by Michael Feathers (deck)
• JVM Performance optimizations at Twitter's scale by Attila Szegedi (deck and video)
• Product Engineering by Mike Lee (deck)
• What the Heck is an API Platform Anyway? by Gregory Brail (deck)
• Building a Great Web API by Evan Cooke (deck)
• Dart in Depth by Gilad Bracha (deck)
• Objects, Anomalies, and Actors: The Next Revolution by Steve Vinosky (deck)
• Things I Wish I'd know by Rod Johnson (deck and video)
• Techniques for Scaling the Netflix API in the Cloud by Daniel Jacobson (deck)

That's all for 2011! We can add to the mix Barcamp, which we sponsored in BA, and then several people in Zauber went to Rosario and Parana to participate of the other Barcamps; also the Google Dev Day here en Buenos Aires.

Now 2012 has started and we are already planning which conferences we will attend! So if you have been to these conferences and want to add other talks that we missed, please comment!

Some thoughts on Agile Development

Last October we attended to the 4th Latin American Conference on Agile Methodologies

This event involved over more than 20 talks and workshops per day delivered by local, regional and international experts.

Here I focus on the main subjects treated and give you some tips you should not miss. Hope you find them interesting and useful!

Story Writing

Sessions in general were focused on determining a better product design figuring out the “why and who” is interested in a particular story.

Jeff Patton, who was in charge of the first keynote in the event, explained a technique used behind this concept: Story Mapping. Fortunately, and besides the amount of people who wanted to participate, we were able to attend to his workshop and gain some practical experience.

Jeff's vision about Scrum and Agile Methodologies in general is that they tend to enforce “output” during a project, i.e. deliver the biggest amount of functionality in a time-boxed period and finish stories already predicted in the backlog for that sprint. Nevertheless, he points out that's not the mean of a successful project. The goal is to satisfy the client (or the co-worker as he wants to call him/her) resolving the first needs that led him/her to actually desiring the product. The only way to accomplish this is by focusing on the project's ‘outcome’. In order to build the best product according to clients' expectations, it's necessary to work on a good release plan, understand the business and break the client role into the different stakeholders that will be taking part in the product.

Story Mapping consists in building a business flow, identify more detailed steps in the flow and establish a hierarchy between the more global steps which are going to become stories and their detailed tasks or sub-stories. Finally, the goal is to assign sub-stories to each release, mapping them with each stakeholder profile for whom the release is planned. Jeff emphasized on how important is the visual, self-organized and cooperative way of working with this technique, especially when human understanding is involved.

In my opinion, you should apply this technique in case you have the opportunity to decide on the product design. This happens in every software factory but outsourcing is not as far as you may think. If you have the chance, it's only about taking the initiative to participate and making the difference.

Regarding Zauber, it's good to confirm that we follow the right approach, getting involved together with our clients in product design and definition, acting not only as a development team, but also as product managers.

Estimation

There were some workshops introducing well known estimation techniques such as PERT applied effectively at enterprise level. Beyond the method itself, it is actually a good practice to think about 3 possible estimations for each story: the optimistic, the pessimistic and the realistic one. This is hard to incorporate but it would help a lot not only to achieve more precise estimations reducing the chance of sub and over estimation but also to identify the very true risks of the story itself.

On the other hand, there were discouraging talks about estimation. These theories are based on the new mixed methodology Scrum-ban which is the combination of both Scrum and Kanban. The new concept, following the story-points idea, is to estimate in stories' complexity or size and build a sprint according to the already known velocity team and the number of stories that can be delivered, without considering the usual hourly-men or ideal days unit. Even though this is a good approximation for reducing projects deviation and leading to a more comfortable work environment, I can't still figure out a way to apply it because clients are not yet prepared for this vision. People in general are used to work with time as their unit measure and it takes a long time to change from one paradigm to another. However, this is a nice theory to follow up and keep in mind.

Continuous Delivery

The talk exposed a truly typical problem concerning a complex and distributed architecture with frequently deploys. Features were easy to develop but integration tests and deployment consumed proportionally a lot more time leading to client’s upset and late delivery of functionality.

The team in charge decided to carry out an automatic system for test and deployment. It took them 1 month development and let them reduce their 10hs delivery time to some remarkable 20 minutes. During the talk, they explained the most important features to take into account when using continuous delivery and finally showed a demo.

The technology used for the project was: Jenkins + Plugin Promoted + Amazon. They casually mentioned they hadn't had good results with Cargo (another known plugin for Jenkins).

There was also an introduction to PAAS (Platform as a service), in particular they showed a little bit of Heroku and its main differences with Amazon.

Our advice is that, if you are about to develop or you're already maintaining a big project, you should adopt continuous delivery and choose the most suitable infrastructure between all the alternatives the market brings, taking into account your non-functional requirements.

I hope you've found this review interesting. If you want to know more about how we build software in an Agile way at Zauber, you can check Agile Software Development

.

Gnip4j: our open source Java library for Gnip

We are happy to announce the public release of Gnip4j, a new Open Source project developed at Zauber and released under the ASFL 2.0.

Gnip4j is a Java library to access Twitter premium feeds.

For more than a year, we've been developing real-time interactive data visualizations, based on social media, gathered mainly from Twitter. We have developed projects for the World Cup, the Brazilian Elections, to track the uprisings in Middle East and more. So far, we always used Twitter Streaming API with granted access to the Gardenhose.

But the requirements of new projects and our ambition to create even more complex and accurate visualizations and real-time dashboards, made us start considering new data feeds. That's why we are now working with Gnip4j and using their premium feeds to take advantage of Twitter firehose to process 100% of tweets.

Our Gnip4j library is the result of our work using Gnip services and we are happy to contribute it as Open Source software, looking forward that other developers can use it and contribute back.

You can get involved by following the instructions here.

Please give us your feedback and feel free to contribute back.

Getting started with Usability and User Centered Design - Part II

Some time ago we detailed the first steps to start including User-Centered Design in your software development process, presenting to you the most easy-to-integrate techniques, but there is more than that if you want to get the best out of User-Centered Design

User research

Doing user research is the core of User-Centered design because it's the best way to understand the user goals, needs and context. It's hard to integrate it into a company's design & development process, but once you get started paying attention to your application's users, guidelines are not enough: you need to know better your audience, and the best way to do it is through a research process.

Ethnographic Interviews:

explore your user's world from an anthropologic point of view. Make interviews registering the context around them, so you can learn about their daily life, how they use their workspace, their problems and needs, how they think about technology, how they use similar and related products. It's very enlightening to learn about real problems of your potential users, once you have do it, you'll probably start thinking in possible solutions and design directions. But first you need to systematize what you've learned

Personas construction:

summarizing what you've learned in Personas will help you to communicate easier your findings about your potential users and their different needs and goals. It's a design tool: all the team can benefit from having fictional users extracted from the interviews. With Personas, people working on the project start talking about well-known characters instead of talking about "the user" without knowing who he is.

Scenarios building:

to build an escenario is to put your Personas in action, narrating the situations in which they will use your product. It's a creative task that helps designers to see the process from the user's perspective (with their goals, needs, and context) and to improve it as well. It also helps them to see the big picture and to be focused in the most relevant tasks. This tasks could also be split into user stories to integrate this phase with the scrum process

Requirements and Use Cases:

Based on the scenarios you can start extracting the most important requirements and building Use Cases. The main difference with other processes is that you have a clear path to follow based on your research.

Usability tests:

once you've designed some screens, you can conduct this kind of tests by asking some potential users to follow a set of predefined tasks in your application or site. You''ll be able to find problems and concerns they might have by observing them while they execute the tasks you asked for. This technique helps you to known better how real users understand your site and what sections of your product needs to be improved or changed.

Card Sorting:

This technique can be useful to give an structure to your site that reflects the way your potential users think about it. You simply need to ask them to organize a bunch of labeled post-its, in a way that makes sense to them. Then you can compare the different answers to find coincidences.

A/B tests:

A/B tests are useful to optimize a section of your application by testing two or more different design options against a large number of users. Then you can know which one works better by analyzing conversion statistics.

Recommended reading

After this short introduction to User Centered Design, you can keep on learning by reading some of the most important books on this subject

• Design of Everyday Things, by Donald Norman, is about the philosophy and basic rules of Usability. Actually, Don Norman isn't a designer, he is a psychologist, and the first edition of this book was called "Psychology of Everyday Things" [Amazon]
• Don't Make Me Think, by Steve Krug. Althought it's a bit old, this book explains in a very practical and easy way how to design a site thinking of users. [Amazon]
• About Face 3, by Alan Cooper, is one of the most detailed books about the whole process of designing an usable product. [Amazon]
• Emotional Design, by Don Norman, is a book about how emotions and aesthetics can be part of Usability that is a very rational approach to design. [Amazon]

Staying in touch with the community

It's better if you have someone who can guide you through this process, but... how to know specialists in this area? IxDA is a global community of interaction designers who take care of usability (among other very related subjects like information architecture, interaction design, user experience and accesibility). It has a lot of local branches that organize events and can help you through your process. We are assisting to events organized by IxDA Buenos Aires

If you want to know more about this subject, you could also visit our User Centered Design section.

Zauber @ QCon San Francisco 2010

This month we participated in the 4th annual QCon San Francisco. This conference is held by InfoQ in several cities all over the world, and the SF edition was a meeting point for over 80 speakers and 500 attendees. It included several tracks related to software architectures, agile software development, IT operations, NoSQL, applications security and more.

It was really hard to choose between 5 different tracks at the same time, mostly because they were all interesting.

You could find people from all over the world, most of them highly technical developers who wanted to learn state of the art techniques. It was amazing to find out about architectural solutions from Facebook, Twitter, Netflix, eBay to keep their systems up serving millions and millions of users.

I chose most of the software architectures and NoSQL talks and I wanted to share some highlights about them:

• Twitter has many problems scaling data, so they went through a series or incremental iterations until they found a good solution using MySQL with partition + replication + indexes + memcache. Also they are currently using NoSQL for data mining, geo indexing and real time analytics. As Ryan King suggested in his talk, there is no database that fills all the gaps, because they tried to use NoSQL to store tweets and they didn't succeed. Ryan also talked about some ad hoc frameworks developed by twitter, like Gizzard, Haplo, Rainbird, Cuckoo and Snowflake.
• Facebook showed how they scale their social graph using MySQL and PHP. Their secret is using MySQL not as a RDBMs but as a schema-less key/value datastore, and compiling PHP to C++ (yes!) for better performance.
• eBay has several systems and thousands of machines running their business. Randy Shoup explained many good practices for addressing problems at that scale like partition everything, asynchronous connections everywhere and incremental system change.
• Netflix is running almost all their infrastructure in the cloud. That was a big shift in their paradigm because they moved from a company that rents DVDs to a company that streams content. Adrian Cockcroft said they are working close to Amazon because they are pushing AWS to its limits!. He also suggested several "cloud patterns" like no sticky session, no chatty protocols (use Apache Avro) and NoSQL.
• Jason McHugh from Amazon S3 demoed how they can handle 30000 request per second on S3. His solution includes distributed hash tables and two level caches, one in webservers and the other over storage, both communicated using a protocol that allows high traffic distribution in order to avoid hot spots. All these solutions were described as resiliency techniques.
• Ryan Dahl, node.js developer, gave a great talk about server performance and asynchronous purity to build scalable network programs.
• Erik Meijer blow up our minds demonstrating that SQL is equivalent to NoSQL. This was the first time he talked about this out of his lab, so it was an inspiring moment.
  
  
• LMAX described how they can handle over 100k concurrent transactions per second with less than 1ms latency. They even optimized code so it can be stored efficiently in CPU caches, and worked with ring buffers for processing huge amounts of transactions. Other techniques included reducing I/O, removing concerns from code and designing a perfect object model.
• Ron Bodkin from Quancast described how they can process 300 billion events per month using Hadoop + Map/Reduce. After that they do data mining with Greenplum. Also he gave a good piece of advice: "start with small datasets so you can understand what is going on, then scale"
• Ralph Johnson, one of the famous "Gang of Four", gave a talk about Parallel Programming Patterns. He talked about Fork-Join (hopefully deterministic, no I/O, no locks), Data Parallelism (deterministic, i.e. Map/Reduce) and Actors (not deterministic, good for I/O)
• Java 8 is going to include project lambda, so Alex Buckley from Oracle explained how Java is going to take advantage of multiple cores.

Also there were four keynotes:

• Patrick Copeland (Google Director of Engineering): this keynote was really inspiring. Patrick talked about the innovation culture at Google and how they develop products. All his ideas are expressed in the Pretotyping Manifesto.
• Dan Ingalls (Principal Architect of Smalltalk): Dan taught us how you can do incredible things and have fun developing software. It was a pleasure to see one of the founders of software as we know it.
• Randy Shoup (eBay Chief Engineer): in this keynote we learnt best practices on being elastic in the cloud.
• Martin Fowler (Chief Scientist at ThoughtWorks): He started talking about his most recent research: DSL. After that he talked about two of his most well known patterns: Continuous Integration and Event Sourcing. Martin is a great speaker and engaged the audience with his huge experience and anecdotes.

As a conclusion, I found some recurrent themes in many talks:

• Using the right tool for the job: at high scale projects you have to try and try different solutions, keep your mind open and choose the right tool. Here is where engineering shines and you can find state of the art solutions.
• Eventual consistency: almost in every talk I heard that phrase. New data models are arising and consumer are multiplying everywhere, thus the web is pushing databases to its limits and here is the price we have to pay. I predict exciting times ahead related to NoSQL vs. SQL and distributed data processing.

Being part of QCon SF 2010 was an incredible experience. Congratulations to InfoQ and QCon for this great event. Zauber will be there again next year for sure!