An straetegy to easily add new tasks to a big project

The authors of Managing the unmanageable came to visit our office and they gave us insights to improve our processes as well as some methods to organize projects. Something that is very interesting about the book, is that it contains valuable and innovative information about managing software projects, and in my opinion the strategies and recommendations can be applied to lots of projects, even if they are not related to software. The one that I'm about to talk about is -I believe- designed for big projects. So let's start talking about the context, so you can find why doing this will be useful and probably the ideas that brought this method into something real.

When you're getting started with a big project, you'll create a first list of tasks that will be developed, from this list, there will come more and more tasks as the project gets developed, so, figuring out how much effort is required for each task can take a lot of time if you don't have anything that leverage the way that this happens, so I'll present you with the steps required for this strategy:

• Divide the project in tasks
• Assign each task a number that represents how difficult the task is
• Sort the tasks

Once that the tasks are sorted, as you can probably imagine, it's very easy to keep the list sorted when more tasks area added to the project, and to estimate the effort required for them,  you just need to compare them with a similar task. This are some of the advantages that you can find when you use this method:

• Adding new tasks is easy, you can compare similar tasks and make a good bet on how much effort it's required for the task and where in the level of effort required does this task falls into place
• You can give a more accurate delivery date for the project, because it's more easy to develop
• Adding big pieces to the project should take less time to organize and make the estimates

Software application architecture - An overview of major software aspects

Software application architecture (I will deliberately use architecture from now on)  is an aspect that should involve the whole team, from the business owners to the developers and the final users -at least a prototype of them-. I will use the MSDN definition because I find it complete and easy to understand: Software application architecture is the process of defining a structured solution that meets all of the technical and operational requirements, while optimizing common quality attributes such as performance, security, and manageability. It involves a series of decisions based on a wide range of factors, and each of these decisions can have considerable impact on the quality, performance, maintainability, and overall success of the application.

Now that we know it's definition and the usage it has, let's talk about key factors related to architecture:

• Communication. I'm not just talking about "being available" for answering questions, the business owners must communicate new features required as well as bugs found in the business logic. They must provide detailed and specific descriptions of the process that the developers are going to work. This also apply from the developer to the business side, if the developers find a feature very hard to develop, they must suggest something and make it clear to the business owners, doing so may provide a different way to tackle the problem
• Documentation. The software system must be documented and described in detail. This documents are the contract that will prevent misunderstands and will help new members of the team understand the system.
• Keeping a list of concerns, assumptions and expectations. This can help when reviewing the project in the future and understanding the architecture decisions. It's a good idea to keep it organized by subject. For example, divide it in infrastructure considerations, software concerns, business analysis, etc.
• Create more than one architecture. It would be very hard to have one architecture, ideally you will create more than one with enough detail to start testing and see how things are looking. It's better to do it iteratively and in an incremental way to refine the architecture.
• Have a due date for completing the architecture. Because architecture involves the whole team, it could take forever to have a "perfect" and approved architecture, but the reality is that we will need to make adaptations because we cannot know everything up front, but we can create an architecture that is ready to handle future changes. So it's a good idea to decide on how much time is the team taking for defining architecture, and start working right away to find issues in an early stage of the project and address them as soon as possible. 
• Take an active part in the decision making and then follow through!. It would be very important that everybody gets involved according to their role in the project. Once that the decisions are made, they should be followed. For example, if you need a web service for some sort of operation and the team decided that X web service is the right one, then it should be used whenever that operation is required. Also if the architecture requires to do some code files in order to create a process, they should be created whenever a new process is requested. On the administrative side, there is also an effect, if the team decided to use an excel spreadsheet for uploading information, the business side must accept this responsibility and create the required files.
• Hide details of implementation. When you're working with a software system, it is important to have a good service layer to reveal only what is necessary. Revealing architecture to other systems can be dangerous for security and in the "best case" it can make other developers think a lot on things like: "why have they do that?". Well... that's not the concern of people connecting to your system, they should only know how to obtain exposed data required for other systems.  

Important factors affected by the architecture:

• Time for adding features. If we got a good architecture, then the new features are pretty much "expected" and they can be delivered on time. This is because we would probably leave the "skeleton" ready for the upcoming features of the system. Besides, when you have a bad architecture, it's really hard to obtain time for re factoring,because it's the most technical aspect of software development, so the other areas involved will not clearly understand the importance of the architecture, so they will not allow much time for re factors.
• Performance of the system. If we do the right choices, the performance will be the expected. This is related to the entries of performance: Back end and Front end.
• New people in the team. If there are new business owners or new developers, a good architecture will make the learning path very straight forward, and this will save time and money to the company.
• Integration with other systems. Most systems nowadays interact with other systems. Having a good architecture (for example, creating a web service layer) will make it easier to interact with other systems, and it's important to see this on the beginning of the project. Doing so will save a lot of time, when this kind of interaction is needed.
• Bugs. Good architecture leads to less bugs, bad architecture leads to a lot of bugs. That's something easy to understand, because if the architecture is not ready to show good messages whenever you're having a new issue, it's very unlikely that you'll find the reason of the bug, also if the architecture has bar practices like not having a good structure of the project and then creating a lot of methods for the same thing, you'll probably find the same bug repeated in different areas of the system.
• Value. The architecture plays a very important role in the value of the project. In the end, the expected times for adding features, the performance, the usability, the results and everything related to the system is directly derived from the architecture decisions, and the users will take advantage of the system, if and only if our decisions are appropriate.

It's complicated to give and estimate on how the architecture affects software systems, it depends on a lot of factors and the context of the systems, but I will use a piece of an interesting article to give you an idea:

"It can be hard to measure the cost of software erosion and convey this cost to non-technical people who often have to sanction work to stop software erosion. Even though software erosion causes reduced productivity, reduced quality and increased time-to-market, no one specific point of erosion causes these effects in isolation, rather it is the effect of multiple points of erosion that combine and reinforce each other to cause them.

However, a study by the US Air Force Software Technology Support Centre (STSC) attempted to put some rough measure on the costs of software erosion. The researchers took two versions of a mature software system (50k LOC) and asked two different teams to perform the same maintenance task (adding approx. 3k of code) on their respective version. Version 1 was an existing system suffering software erosion. Version 2 was the same system but with the architecture restructured to remove erosion.

The results were staggeringly different. Team 1, working on Version 1, needed over twice as long as team 2 to complete this relatively short task. Furthermore, Team 1’s results contained more than eight times the number of errors than the work submitted by team 2, working on version2. Erosion in a small system such as this still had the potential to lead to significant problems when the software was maintained."

This is the link to the original article:

http://www.methodsandtools.com/archive/archive.php?id=85

Ok, this is just the tip of the iceberg, but I wanted to share with you, how important is to have a good architecture. My best advice is to do early planning and to have great communication channels between teams, this are the very key factors to success. I hope you're enjoying the series! if you want me to talk about a particular software topic, let me know in the comments.

Back end Performance - An overview of major software aspects

I decided to start with back end performance, because I believe it's the most technical side of software development, and it's a topic that every software developer must know about. I will start with some definitions, that I'm going to use in the post to make it easy to understand, keep in mind that we're in the context of computer sciences and systems, you will need to have some background in algorithms, and data structures:

• System Performance. In a few words, how effective your system is in the desired scenario. There are two scenarios, one is how much time does your system requires to execute a task, the other one is how much memory are we using for executing a process.
• Computer speed. How many operations a computer can execute in a unit of time. Usually this depends on the task, and the type of operation. For the examples, we will suppose that a computer can execute N number of operations in a minute. 
• Data structure. Is a particular way of organizing data (information).  
• Algorithm. It's a finite, optimal, complete, accurate, precise, step by step definition for solving a problem. Some people doesn't include finite or efficient as part of the definition, but I want to include them because if it's not, then it's not useful, an algorithm is expected to be fast and intelligent.
• Complexity. How efficient an algorithm is in a field. In the case of speed, it's measured in the number of atomic operations required for the algorithm to return the data we will call this speed complexity. The memory is measured in the amount of memory used for the algorithm to return the data we will use memory complexity when referring to this field. Both complexities are set against the size of the entry. For example, if the parameter for the algorithm is an array of size N,  then, the speed complexity is how many operations do we have to do against that array  to obtain the desired result (N, log(N), etc). For the memory is the same history, how much memory do we need to execute the algorithm apart from the array passed as parameter. Because memory is growing faster than processor speed, we prefer to use more memory and less processor when we can.
• NP (complexity class). It's a class of problems that cannot be solved in polynomial time.
• Heuristic: It's a step by step definition for solving a problem, the difference with algorithms is that it doesn't have one or more of the characteristics of the algorithms: finite, optimal, complete, accurate or precision. It's an approximate solution that speeds the process. For example, organizing a list of tasks in a way that they will be solved as fast as possible, these problem falls in the NP complexity class, and is faster to use an heuristic to solve it.

This aspect is very important, because even with all the improvements to the memory and processor speed, having an effective algorithm for solving a problem is the only way to keep things fast. Have you ever notice how much does the processor speed increases every year? Whatever the answer is, the truth is that you'll see that if the old speed was X, the new speed is expressed as NX, i.e. the speed was increased by a linear factor. What will happen if we use algorithms that have a speed complexity beyond that linear factor? We'll answer that in this example:

Example: Comparing binary search against linear search

Execution information:

• We will suppose that we've two computer:
• Computer A. Speed: 100 operations per second. It will use linear search.
• Computer B. Speed: 1000 operations per second. It will use binary search.
• We will execute 100 search operations.
• The data structure is going to be an array containing 20,000 elements sorted alphabetically. 
• We will suppose that for each binary search we will need 9 operations and for linear search we will need 10,000 operations. I'm suggesting this, in order to have a way to compare the execution times of the algorithms. Worst case for binary search with that size is  15  and for linear search is 20,000.

Execution time in seconds for each computer:

• Computer A: We will search for 1000 elements, and for each one, we're supposing that the binary search algorithm will require 9 operations, so the total number of operations is (1000 * 9 = 9000. We will require 9000 operations to complete or test, and this computer can execute a total number of 100 operations per second, so it will take 9000 / 100 = 90 seconds to complete the task. 
• Computer B: Using what we already explained for computer A, the execution time in seconds is: (1000 * 10000) / 1000 = 10000.

Are you surprised? Computer B can execute 10 times more operations per second in comparison with Computer A. But as you can see, if we use a slow algorithm, the execution times are really high for Computer B! this is the reason to optimize algorithms and keep things fast. 

Can you think about another reasons to keep things fast? check this list:

• Hosting prices. If you have a web system that is using bad algorithms, then it's very likely that you'll need to have more a better processor and more memory to cover the demand of your web system. This is traduced in more money each pay period!
• Compatible devices. If your app needs more processing speed, then your app is not going to be compatible with all devices, this means that your market is going to be reduced and some people will use other app that can be executed by low-end devices. And also, remember that memory in small devices is a constraint. If your app needs a lot of memory, only top end devices are likely to run your app.
• Better graphics. The faster your algorithm is, the more beautiful the graphics are going to be, because you'll use more processor and memory for displaying them.
• Battery life in mobile devices. Slow algorithms use more processor, and the processor is one of the thing that uses more battery! so try to keep things fast when developing a mobile app!

Something to keep in mind is that this is just the tip of the iceberg. In order to have great performance, the context of your application is also very important, for example:

• If your have a process that takes much time to return a set of data, but you know that it only changes every day, you can have a cache of data and query it when a search is taking place. 
• If it's more important to bring data faster than accurately, you can use a flag to set the required quality of the data returned or you can retrieve some data while you process the best result and then return it. 
• If your algorithm can be a multi-theaded algorithm, then do it, most devices nowadays have more than one core processor.
• Do you need the user to wait until the process is finished? for example, let's say that your app is sending a request that needs to be verified in some other systems, that could take up to 5 minutes to complete. In this cases, is better to send the required data, and let the process work, and tell the user something like: "your request is in process" and close that dialog and let the user work with the app. When the process is finished, then you've to send a message with the results.
• Always keep an eye on the end user's opinion, requirements, and needs. It's worth nothing a system that does some features faster than another if the user is using more the slow features.

Ok, that's it! those are the reasons to always keep an eye on the performance of your system. This is the first entry of the series, I hope you enjoy it. If you want me to write about some particular subject or topic, please let me know! I'll be happy to do that.

Btw, this is the index blog post, here you can see all the posts of the series and some background on what to expect:

http://codeauthorityandmanagement.blogspot.mx/2014/09/index-overview-of-major-software-aspects.html

Being extremely successful at working remotely

Working remotely can be easy and awesome, you just need to realise that it is way more different in comparison with working in an office and also use a lot of common sense to be effective and successful.

These are some conditions that are -almost- always true when working remotely:

1. You need to be available for meetings
2. Answering and reading email is important
3. You'll need a good internet connection
4. A second monitor is a must
5. You'll be at least in one meeting during the day
6. You'll do less exercise because your home is your new office

I will present recommendations based on the given conditions, but you can always evaluate your own conditions and set up your work rules to be productive:

1. Stay visible! Make sure that during your work hours you're always reachable. Log into your IM account and stay available during work. I suggest to not spend time messing with the IM client status. Current chat clients have ways to identify if you've been away for a period of time enough to put you in a "away from computer" status. And also, changing your status to "busy" can be bad, why will you change it? if you are going to be really busy, and you're not being available for a period of time, you can log off and let everyone know the reason. On the other hand, if you're again "available" will you remember to change the status again? if you don't, then the members of the team could think that you're busy and they'll not contact you
2. Have a comfortable and effective work place. Let's talk first about furniture: Make sure that you have a desk with enough room for your hardware requirements. I suggest to have enough room for a laptop and a second monitor, another important piece of furniture, is a comfortable chair, make sure that you don't hurt your back or neck with it. Talking about an effective place, you've to take into account that you're probably being in meetings during the day, so I suggest to have a quiet place or make it quiet with a noise cancelling curtains for example. another important piece for this is a good headset, a "trick" that many people don't know is that you can probably use the headset of your cellphone in your laptop, those devices are usually great, good looking and using them can save money.
3. Be communicative. Are you about to finish a task? Have you been working a lot with a co worker? Are you changing your lunch schedule? All of these situations needs to be communicated to the team. For example, if you're about to finish a task, you'll probably need to pair with a member for reviewing or actually, your manager will be assigning a new task soon depending on how much time you're planning to spend on the current task. It's a good idea to let the members of the team that are actively working with you know when you're grabbing lunch. 
4. Be ready for stand up. Prepare a couple of minutes before stand up: check your items, your needs, open the meeting, go to the restroom, bring a cup of tea or coffee (in case you enjoy them!)
5. Email and chat latency. Set email and chat notifications to "on" and answer them quickly. For chat in less than one minute. For email, well... depends, but ideally in less than 10 minutes, sometimes email can be a little more complex. And also, don't forget to check all the emails you receive,  no matter if they're a general announcement, you'll probably want to know if the office is being closed for a couple of days :)
6. Schedules. Try to decide on an appropriate schedule. Set your work hours and lunch time and let everyone know. If you're going to change something for a reason, you should always send a message!
7. Cameras. Yes! video chats are really awesome when working remotely and they're so overrated. Seeing the people of the meeting will help in understanding the point of view of your team, you will feel closer to them and then you'll communicate better. Just give it a try! it really improves overall communication. 
8. Remote meeting tools. Try to use as much as possible tools like sharing screen, drawing in the screen, letting other person take control of your computer. Record your meetings. These tools will help in being more productive and will improve communication.
9. Set rules based on your own context. Talk with everybody on your team and set the appropriate rules for your team. For example, if a member of the team is arriving earlier than everyone, he could probably do the tasks of database maintaining.
10. The other side of the mirror. For those people on the team working in the office, remember that some rules also apply to you. For example, if you're working closely with someone, it  would be a good idea to let them know about your meetings and the times when you're going to be unavailable, remember that the remote team members cannot go to your place and see that you're not in your spot. In general, it's a good practice to set up a calendar visible to every member of the time with the available and unavailable times of everyone.
11. Do more exercise!  Trust me... you'll gain some weight because you'll be doing much less exercise, so this is the right time to attend some kick boxing classes and stay in shape! -ok just kidding, but really, try to do exercise, running, hiking, martial arts, yoga, gym, whatever you decide-

As I've already say,  these recommendations are based on my experience, every team should sit and dedicate time in creating their own particular rules, these will make them more productive.

I hope you enjoy these entry! feel free to leave your comments at the end of the post.