A very small introduction to hadoop and map reduce.. Must read article

Hadoop was created by the Apache foundation as an open-source software framework capable of processing large amounts of heterogeneous data-sets in a distributed fashion (via MapReduce) across clusters of commodity hardware on a storage framework (HDFS).  Hadoop uses a simplified programming model.  The result is Hadoop provides a reliable shared storage and analysis system.
MapReduce is a software framework that allows developers to write programs that perform complex computations on massive amounts of unstructured data in parallel across a distributed cluster of processors or stand-alone computers.  MapReduce libraries have been written in many programming languages (usually Java), with different levels of optimization.  It works by breaking down a large complex computation into multiple tasks and assigning those tasks to individual worker/slave nodes and taking care of coordination and consolidation of the results.  A MapReduce program is composed of a Map() procedure that performs filtering and sorting (such as sorting students by first name into queues, one queue for each name) and a Reduce() procedure that performs a summary operation (such as counting the number of students in each queue, yielding name frequencies).

Hadoop Distributed File System (HDFS) is a distributed file-system that stores data on commodity machines, providing very high aggregate bandwidth across the cluster.  When data is pushed to HDFS, it will automatically split into multiple blocks (128MB by default) and stores/replicates the data across various datanodes, ensuring high availability and fault tolerance. 

NameNode holds the information about all the other nodes in the Hadoop cluster, files present in the cluster, constituent blocks of files and their locations in the cluster, and other information useful for the operation of  the Hadoop cluster.  Each DataNode is responsible for holding the data.  JobTracker keeps track of the individual tasks/jobs assigned to each of the nodes and coordinates the exchange of information and results.  TaskTracker is responsible for running the task/computation assigned to it.

Trends in software testing and predictions..

Software Testing Trends and Predictions – 2015

The year 2014 witnessed cross border mergers, acquisitions and rapid adoption of social media platforms. It also witnessed creation of large chunks of data and rapid advancements in mobile and cloud technologies. Another major advancement was Internet of Things (IoT), which is slowly changing the way how organizations interact with their customers, employees and partners. In the current digital information age, individuals are demanding improved connectivity and on the go access to information, which is making businesses, Government organizations and other public sector entities to be more agile and flexible while offering services and deliverables.

The rapid influx of new technologies has thrown open new set of challenges to the present day CIOs, increasing their responsibilities towards delivery and support. Business leaders are expecting CIOs to come out with innovative ideas and outline a vision on how emerging technologies can shape up their organizations. Today’s CIO is also expected to help organisations to understand customer preferences in a better way and help them retain customers, along with new acquisitions. The year 2015 is going to be a huge challenge for the software testing fraternity due to emergence of new technologies, let’s take a look at some of the latest trends in the software testing domain for the year 2015.

Software Testing Trends 2015

In 2015 as well, the top priority of businesses across the globe remains Cost Optimization – delivered through process and technology. Additional priorities include supporting organic growth, build new products and platforms and upgrade their existing systems. Most of the IT leaders accept that integration of existing IT systems is a top priority for their organizations. Digital transformation remains another important focus and businesses are ranking high on cloud computing and business analytics.

Quality and reliability are being given greater attention, which is resulting in reduction of software application errors, improving security and application performance. Social media is being closely scrutinized by organizations in order to safeguard from reputational damages that often result from the comments posted on organization’s social media channels. These testing trends would also have a significant impact on the discipline of quality assurance and software testing in 2015. I would like to give due credit to Gartner and IDC for providing valuable foresight on software testing industry trends for 2015 and beyond.

Software Testing Industry Predictions for 2015

Transformational Projects Will Drive QA and Software Testing Budgets

Software testing budgets will be on the rise, especially in the sectors of energy & utilities (E&U) and transportation, these domains will most likely be spending 27 percent and 31 percent of their budgets on QA and testing. It has also been observed that most of the organizations are allocating more than 40 percent of their IT budget towards quality assurance and software testing domains.

Testing budget allocated for new software development initiatives is now slowly overtaking budgets that are usually allocated for software maintenance projects, which is a positive sign. Consumer products, retail, financial services and transportation sector are swiftly advancing towards new IT development initiatives. IT trends such as big data analytics, cloud, mobility and virtualization will continue to grow, although these are considered as fairly expensive technologies to develop and maintain.

Testing Center Of Excellence (TCOE) – Continues to Grow

Another trend in testing is the concept of TCOE, completely operational TCOE models have effectively grown to 26 percent from a mere 4 percent in the year 2011. A TCOE can be present in multiple forms, it can exist as a pure methodological support, which will control QA and testing or can exist as a complete test execution factory, which will provide on demand testing services to software projects across the organization. Businesses are also looking to set up TCOE’s using a combination of best practices to offer improved governance, process, control and metrics.

Crowdsourcing – Good or Bad?

Crowd sourced testing a relatively new trend in software testing, where software applications are made available to a large number of end-users, who will try to identify defects and bugs. Crowd sourced testing services are usually managed by companies ‘as a service’. Interestingly, almost 27 percent of organizations have already experimented with the concept of crowd sourced testing and are willing to continue with this new software testing trend. But I believe that crowd sourced testing can never replace the conventional quality assurance and testing teams.

2015 is for SMAC Technologies

The increasing usage of social media, mobile, analytics and cloud technologies also referred together as SMACimproves operational efficiencies and strengthens relationships with customers and end-users. There is a significant demand from customers, employees and end-users for responsive, anywhere accessible software applications that improve the areas of communication, business and entertainment. The increasing usage of social media is creating “Zero Tolerance” environment, as any failure, error or outage is resulting in immediate exposure to the global audience via social media channels. Organizations will be spending 52 percent of their IT budget towards QA and testing, out of which major allotments are 27 percent for cloud, 40 percent for big data & analytics and 17 percent for mobile.

Mobile, Mobile, Mobile

Mobile devices have become a part and parcel of our daily lives and the same goes for businesses. Mobile technologies have established themselves as an effective profit making assets for an undeniably disseminated and on the go workforce.

Mobile Application Testing

The mobile application testing trend is going to have a major impact on QA and software testing industry. There has been is a steep rise in organizations that are practicing mobile application testing, which stood at 31 percent in the year 2012, but increased to 55 percent in 2013 and 87 percent in the year 2014 and it is not an exaggeration, but the figures may actually rise up to 95 percent by the end of 2015. Functionality testing, security testing and performance testing are the primary focus areas for organizations that are involved in mobile application testing, which are followed by usability testing, regression testing and compatibility testing. Gartner predicts that there will be about four Android devices for every Windows or Apple (iOS) device by the year 2015.

Mobile Testing Best Practices

With the rapid changes in mobile technologies, businesses are expecting matured mobile testing practices from organizations offering mobile testing services. Software testing applications should be able to support a wide range of devices and operating systems, which would help in the creation of multiple test scenarios / executions across multiple platforms. The development lifecycle of mobile applications is relatively compact as compared to a standard or a web application; hence it is always a challenge for software testing teams to complete the testing within the allocated time frame. Almost 40 percent of the organizations agree that time is a crucial factor for QA and testing teams, which highlights the importance of automation to perform regression testing.

PC Gets Boost, Still Outpaced By Mobile

Android Rules, When It Comes To Operating System

Big Data and Analytics

Big data and analytics testing is a catching up trend in software testing domain. It is not only about testing vast chunks of data, but it is all about testing the correctness and accuracy of data (i.e. Obtaining a meaningful and actionable information), which is sourced from multiple platforms. As per my analysis, big data and analytics testing will account for approximately 50 percent of the total software testing budget in the year 2015. Big data and analytics testing applications are highly complex and pose a big challenge to the software testing teams.

The software testing teams will be working with data centric processes and would not only be responsible to identify data irregularities and quality issues, but also figure out performance issues, security concerns, accuracy of ETL processes and most importantly verify, if the sourced data is relevant and suitable for business analytics.

Cloud Migration

Despite the odds cloud computing faced in its initial years, it will catch up in the year 2015. Although, we have witnessed a decline in cloud adoption in 2013 (20 percent), but it bounced back and gained momentum by demonstrating a positive trend in 2014 (28 percent) and is expected to reach 32 percent by the end of 2015. Organizations are benefiting immensely by adopting cloud services, especially by building and maintaining cloud environments. 

It would be considerably easy for software testing teams in the cloud environment, if they are provided the right environment and desired ramp up to an agreed load capacity. On the contrary, not all software applications can be hosted in the cloud environment e.g. large enterprise systems, which require robust integration testing would have to be hosted in a separate infrastructure test environments. However, the cloud testing trend is here to stay for a foreseeable future.

Conclusion 

Considering the above software testing trends, statistics and projections, we can confidently predict a bright future for quality assurance and software testing domain, especially in the areas of automation testing, performance testing and security testing. Software testers need to be prepared and be ready to grab the emerging opportunities in the 13 billion dollar software testing industry.

Automated vs. Manual Testing: The Pros and Cons of Each

You can test software through both automated and manual testing, but which one you choose comes down to the associated costs and benefits of each on your particular project. Today we’ll walk you through the pros and cons of using both automated and manual testing methods so that you can determine which is best for your project. Hint though? The answer can be a combination of both.

Automated Testing

Automated testing is the process through which automated tools run tests that repeat predefined actions, comparing a developing program’s expected and actual outcomes. If the program expectations and outcomes align, your project is behaving as it should, and you are likely bug free. If the two don’t align, however, there is an issue that needs to be addressed. You’ll have to take a look at your code, alter it, and continue to run tests until the actual and expected outcomes align.

Automated testing is good to use when the project is large, there are many system users, or when filling out forms.

Pros of Automated Testing:

1. Runs tests quickly and effectively

While the initial setup of automated test cases may take a while, once you’ve automated your tests, you’re good to go. You can reuse tests, which is good news for those of you running regressions on constantly changing code. You won’t have to continuously fill out the same information or remember to run certain tests. Everything is done for you automatically.

2. Can be cost effective

While automation tools can be expensive in the short-term, they save you money in the long-term. They not only do more than a human can in a given amount of time, they also find defects quicker. This allows your team to react more quickly, saving you both precious time and money.

3. More interesting

Filling out the same forms time after time can be frustrating, and not to mention boring. Test automation solves this problem. The process of setting up test cases takes coding and thought, which keeps your best technical minds involved and committed to the process.

4. Everyone can see results

When one person is doing manual testing, the rest of the team can’t see the results of the tests being run. With automated tests, however, people can sign into the testing system and see the results. This allows for greater team collaboration and a better final product.

Cons of Automated Testing:

1. Tools can be expensive

The automation tools can be an expensive purchase. As a result, it is important to only use the ones that will give you full, or as close to full coverage, as you can find.

2. Tools still take time

While the automation process cuts down on the time it takes to test everything by hand, automated testing is still a time intensive process. A considerable amount of time goes into developing the automated tests and letting them run. For example, a large client of ours ran into trouble when their daily run of automated tests exceeded the 24-hour mark.

3. Tools have limitations

While automated tests will detect most bugs in your system, there are limitations. For example, the automated tools can’t test for visual considerations like image color or font size. Changes in these can only be detected by manual testing, which means that not all testing can be done with automatic tools.

Manual Testing

Manual testing is the process through which software developers run tests manually, comparing program expectations and actual outcomes in order to find software defects.

These manual tests are no more than the tester using the program as an end user would, and then determining whether or not the program acts appropriately. Manual testing is a good fit for smaller projects as well as companies without significant financial resources.

Pros of Manual Testing

1. Short-term cost is lower

Buying software automation tools is expensive. With manual testing, you won’t have to put the same up-front costs into the software.

2. More likely to find real user issues

Automated tests are just that – automatic. They’re robotic and don’t necessarily act as a real user would. Manual testing, on the other hand, allows the developing program to be used as it would be upon launch. Any bugs that may pop up when a user handles the program in a certain way are more likely to be caught with manual testing.

3. Manual testing is flexible

When one of those brilliant thoughts comes to you, something that could change the course of the project, you want to be able to work on it immediately. With automated testing this is difficult. You have to set up test cases, program it into the automated tool, and then run the tests. With manual testing, you can just quickly test and see the results. Automatic tests take more time to set up, which doesn’t allow you to test ideas quickly and easily.

Cons of Manual Testing:

1. Certain tasks are difficult to do manually

There are certain actions that are difficult to do manually. For example? Low level interface regression testing. This kind of testing is extremely difficult to perform manually, and, as a result, is prone to mistakes and oversight when done by hand. Automated testing, once set up, is much better equipped to find errors for this kind of testing.

2. Not stimulating

Manual testing can be repetitive and boring – no one wants to keep filling out the same forms time after time. As a result, many testers have a hard time staying engaged in this process, and errors are more likely to occur.

3. Can’t reuse manual tests

With automated tests, if you add anything to the program, you can rerun all of the required tests instantly – the tests are already set up. This isn’t the case with manual testing. If there is any change to the software, you have to run the tests again by hand. This is valuable time lost.

There are both pros and cons to automated and manual testing. While many say that common sense is supposed to guide you in making the decision of which one to use, common sense won’t lead you unless you have all of the information. Make sure that you consider your time, your resources, and the size of your project as well as the quality of the automated tools you’ll be using and your testing team’s knowledge. Once you take these things into mind, you’re more likely to do what’s best for you.  Always remember though that combining both is an option. In fact, combining the two may be optimal for canceling out the others’ cons and developing the best software possible.

Why software testing is necessary ?

Why is software testing necessary?

Since we assume that our work may have mistakes, hence we all need to check our own work. However some mistakes come from bad assumptions and blind spots, so we might make the same mistakes when we check our own work as we made when we did it. So we may not notice the flaws in what we have done.Software Testing is necessary because we all make mistakes. Some of those mistakes are unimportant, but some of them are expensive or dangerous. We need to check  everything and anything we produce because things can always go wrong – humans make mistakes all the time. 

 Ideally, we should get someone else to check our work because another person is more likely to spot the flaws.

There are several reasons which clearly tells us as why Software  Testing is important and what are the major things that we should consider while testing of any product or application.

Software testing is very important because of the following reasons:

1. Software testing is really required to point out the defects and errors that were made during the development phases.
2. It’s essential since it makes sure of the Customer’s reliability and their satisfaction in the application.
3. It is very important to ensure the Quality of the product.  Quality product delivered to the customers helps in gaining their confidence.
4. Testing is necessary in order to provide the facilities to the customers like the delivery of high quality product or software application which requires lower maintenance cost and hence results into more accurate, consistent and reliable results.
5. Testing is required for an effective performance of software application or product.
6. It’s important to ensure that the application should not result into any failures because it can be very expensive in the future or in the later stages of the development.
7. It’s required to stay in the business.

Top 8 Java People you must know ..

8. Tomcat & Ant Founder

James Duncan Davidson, while he was software engineer at Sun Microsystems (1997–2001), created Tomcat Java-based web server, still widely use in most of the Java web projects, and also Ant build tool, which uses XML to describe the build process and its dependencies, which is still the de facto standard for building Java-based Web application.\

7. Test Driven Development & JUnit Founder

Kent Beck, creator of the Extreme Programming and Test Driven Development software development methodologies. Furthermore, he and Erich Gamma created JUnit, a simple testing framework, which turn into the de facto standard for testing Java-based Web applications. The combine of JUnit and Test Driven Development makes a big changed on the way of coding Java, which causes many Java developers are not willing to follow it.

6. Java Collections Framework

Joshua Bloch, led the design and implementation of numerous Java platform features, including JDK 5.0 language enhancements and the award-winning Java Collections Framework. In June 2004 he left Sun and became Chief Java Architect at Google. Furthermore, he won the prestigious Jolt Award from Software Development Magazine for his book, “Effective Java”, which is arguably a must read Java’s book.

5. JBoss Founder

Marc Fleury, who founded JBoss in 2001, an open-source Java application server, arguably the de facto standard for deploying Java-based Web applications. Later he sold the JBoss to RedHat, and joined RedHat to continue support on the JBoss development. On 9 February 2007, he decided to leave Red Hat to pursue other personal interests, such as teaching, research in biology, music and his family.

4. Struts Founder

Craig Mcclanahan, creator of Struts, a popular open source MVC framework for building Java-based web applications, which is arguably that every Java developer know how to code Struts. With the huge success of Struts in early day, it’s widely implemented in every single of the old Java web application project.

3. Spring Founder

Rod Johnson, is the founder of the Spring Framework, an open source application framework for Java, Creator of Spring, CEO at SpringSource. Furthermore, Rod’s best-selling Expert One-on-One J2EE Design and Development (2002) was one of the most influential books ever published on J2EE.

2. Hibernate Founder

Gavin King, is the founder of the Hibernate project, a popular object/relational persistence solution for Java, and the creator of Seam, an application framework for Java EE 5. Furthermore, he contributed heavily to the design of EJB 3.0 and JPA.

1. Father of the Java programming language

James Gosling, generally credited as the inventor of the Java programming language in 1994. He created the original design of Java and implemented its original compiler and virtual machine. For this achievement he was elected to the United States National Academy of Engineering. On April 2, 2010, he left Sun Microsystems which had recently been acquired by the Oracle Corporation. Regarding why he left, Gosling wrote on his blog that “Just about anything I could say that would be accurate and honest would do more harm than good.”

10 Programming Languages that you must learn

1. Java

What it is: Java is a class-based, object-oriented programming language developed by Sun Microsystems in the 1990s. It's one of the most in-demand programming languages, a standard for enterprise software, web-based content, games and mobile apps, as well as the Android operating system. Java is designed to work across multiple software platforms, meaning a program written on Mac OS X, for example, could also run on Windows.

Where to learn it: SEED Infotech Ltd., Kolhapur

2. C Language

What it is: A general-purpose, imperative programming language developed in the early '70s, C is the oldest and most widely used language, providing the building blocks for other popular languages, such as C#, Java, JavaScript and Python. C is mostly used for implementing operating systems and embedded applications.

Because it provides the foundation for many other languages, it is advisable to learn C (and C++) before moving on to others.

Where to learn it: SEED Infotech Ltd., Kolhapur

3. C++

What it is: C++ is an intermediate-level language with object-oriented programming features, originally designed to enhance the C language. C++ powers major software like Firefox, Winamp and Adobe programs. It's used to develop systems software, application software, high-performance server and client applications and video games.

Where to learn it: SEED Infotecch Ltd., Kolhapur

4. C#

What it is: Pronounced "C-sharp," C# is a multi-paradigm language developed byMicrosoft as part of its .NET initiative. Combining principles from C and C++, C# is a general-purpose language used to develop software for Microsoft andWindows platforms.

Where to learn it: SEED Infotech Ltd., Kolhapur

5. Objective-C

What it is: Objective-C is a general-purpose, object-oriented programming language used by the Apple operating system. It powers Apple's OS X and iOS, as well as its APIs, and can be used to create iPhone apps, which has generated a huge demand for this once-outmoded programming language.

6. PHP

What it is: PHP (Hypertext Processor) is a free, server-side scripting language designed for dynamic websites and app development. It can be directly embedded into an HTML source document rather than an external file, which has made it a popular programming language for web developers. PHP powers more than 200 million websites, including Wordpress, Digg and Facebook.

Where to learn it: SEED Infotech Ltd., Kolhapur

7. Python

What it is: Python is a high-level, server-side scripting language for websites and mobile apps. It's considered a fairly easy language for beginners due to its readability and compact syntax, meaning developers can use fewer lines of code to express a concept than they would in other languages. It powers the web apps for Instagram, Pinterest and Rdio through its associated web framework, Django, and is used by Google, Yahoo! and NASA.

8. Ruby

What it is: A dynamic, object-oriented scripting language for developing websites and mobile apps, Ruby was designed to be simple and easy to write. It powers the Ruby on Rails (or Rails) framework, which is used on Scribd, GitHub,Groupon and Shopify. Like Python, Ruby is considered a fairly user-friendly language for beginners.

9. JavaScript

What it is: JavaScript is a client and server-side scripting language developed byNetscape that derives much of its syntax from C. It can be used across multiple web browsers and is considered essential for developing interactive or animated web functions. It is also used in game development and writing desktop applications. JavaScript interpreters are embedded in Google's Chromeextensions, Apple's Safari extensions, Adobe Acrobat and Reader, and Adobe's Creative Suite.

10. SQL

What it is: Structured Query Language (SQL) is a special-purpose language for managing data in relational database management systems. It is most commonly used for its "Query" function, which searches informational databases. SQL was standardized by the American National Standards Institute (ANSI) and the International Organization for Standardization (ISO) in the 1980s.