Data Analysis And Data Science: How to Hire for the Right Role

Data analysis is the process of drawing insights from data. On its own, data can be messy, complex, and overwhelming. A data analyst will take that dataset and provide meaningful, actionable insights to the business.  If you’re used to asking, “How many sales did we have on Tuesday?” a data analyst will help provide context and value. How does Tuesday compare to other days of the week? Are Tuesday’s sales lower or higher than other Tuesdays? What else changed that might have affected Tuesday’s sales?

These sorts of insights can be drawn on an ad-hoc basis, but to be most effective, data analysis requires a frequently updating archive of data since studying data from last year won’t tell you the trends of your business today. As such, data analysis also includes the initial steps of intaking data and validating it; and the ongoing steps of making the data and metrics available to decision-makers in the organization. 

A data analyst can be expected to create data intake processes, write and automate data validation scripts, query data on-demand, convert business needs into metrics, and create reporting, dashboards, and data visualizations.  

As with any role, relevant project experience is the most informative element of a candidate’s profile. While transferable work experience is most desirable, one should also look for relevant coursework, class projects, or personal work that might qualify someone for the role.  

Technical qualifications to look for include:

• Database management via SQL, Excel, Oracle, or Access
• Data visualization via Excel, Power BI, or Tableau
• Some programming experience via R, Python, Visual Basic, C++, or SAS

To assess behavioral qualifications, one might consider:

• Communication: Explain a technical project you’ve worked on as if I’m a layperson with no relevant knowledge
• Attention to detail: What steps do you take to ensure the work you do is accurate?
• Self-sufficiency: If I give you this dataset without any context, what insights can you provide?
• Ability to learn: A data analyst is required to work with many different parts of the business. How do you ensure that the deliverables you create properly meet the needs of the business?

The data analyst position can be a great entry-level role for hiring into the company. Recent graduates can be effective contributors to a team, but will usually require direction and oversight from a more experienced analyst. Data analysts can grow over time to serve as data scientists, software developers, product owners, and quality assurance engineers, all depending on their role, the organization, and the employee themselves.

Data science seeks to predict or prescribe behavior based on large, frequently updating sets of data.  At its core, data science seeks to draw more powerful insights from data and utilizes tools and techniques that can accommodate greater uncertainty or datasets too large for humans to reasonably understand.  

To illustrate, a grocery store may have thousands of data points about each of its customers.  Demographic data like age, address, and family size can be fairly simple to analyze.  But once all the variables of shopping data are introduced, things quickly become complicated.  Each customer’s visit - what time and how often they come in - and even their whole shopping list is open for analysis.  Consider a father with two sons in high school who buys power bars every Sunday night - should the store send him a coupon for a 24-pack of sports drink?  Scaling this sort of analysis across thousands of customers, thousands of grocery items, and performing it on an ongoing basis with constantly updating data is too much for even a team of analysts.  

A data scientist can use statistics and algorithms to comb through all this data, create inferences for gaps in the dataset, and create a model to simulate shopper behavior.  This model is consistently tested and updated as new data is received.  While data analysis seeks to answer specific questions to inform business decisions, data science identifies trends and patterns to come up with questions that you may not have realized needed answering.

A data scientist requires a strong background in data analysis, statistics, and programming. Most candidates have a Masters, a Ph.D., or a rigorous set of certifications; data scientists holding solely a Bachelor's degree are exceedingly rare.

Technical qualifications to look for include:

• Strong programming experience via R, Python, or SAS
• A strong background in statistics and modeling
• Familiarity with analytical engines such as Alteryx, Hadoop, or Spark
• Data wrangling via SQL; familiarity with PostgreSQL or mongoDB
• Data visualization via Power BI or Tableau

Even more than a data analyst, a data scientist’s ability to communicate complex technical work to others in the organization is essential. Data science algorithms are complicated and need to be properly informed by the business or industry - if the model’s assumptions are wrong, the output won’t be reliable. A data scientist must be able to both listen and explain with equal competence.

There can be a large area of overlap between a data analyst and a data scientist. Most data scientists have the skills of data analysts, and many data analysts study to grow their data science skills. The key lies in finding the best role for your project.  Just as one wouldn’t hire a construction foreman to lay bricks, one wouldn’t hire a data scientist to create charts for the monthly executive presentation. Depending on the scale of your project, you may be best served by having a team composed of both roles.

Consider utilizing a data analyst to:

• Create a recurring data import process
  Our finance department wants to incorporate individuals’ sales figures into their monthly analysis. How do I bring that data into our system every month?
• Develop recurring reports for periodic delivery
  Finance has their monthly data feed built, but others in the organization want visibility to the numbers as well. Can we build a report or dashboard to keep them informed?
• Implement a data validation process
  Our salespeople manually enter their figures every day.  How do I catch a typo or an extra zero?
• Develop simple predictive analytics such as linear or cyclical regression analysis
  Based on our sales numbers from this year, what can we expect our sales to be next year?
• Build out a database to combine data from a variety of sources
  We have employee data from HR, Finance, and IT, so I always have to look up one employee three times. How can I see all three in one location?
• Perform ad-hoc analyses of clean and comprehensive datasets
  Our sales numbers were especially low last week.  How does that compare to prior weeks or last year? What else might have been going on to cause that?
• Design data visualizations like charts, maps, or reports
  We’re presenting to our CEO on Friday and need to explain why sales in certain states were less affected by the economic slowdown. Can we build a visualization to capture that in a single slide?

Consider utilizing a data scientist to:

• Provide inferences from messy or incomplete data
  Due to state regulations, we’re missing demographic data for the past two years. Using our historical data and other datasets, can we infer what those demographics might look like?
• Develop a model to simulate complex processes or consumer behavior
  What kind of purchasing behavior can we expect from a specific profile?
• Develop complex predictive analytics
  An automotive manufacturer is releasing a new model next year. To inform the manufacturing division, how many of the new vehicles should they expect to sell, accounting for the economy, competitor models, and segment popularity?
• Create machine learning algorithms to identify trends and patterns
  An airline conducts thousands of flights each day between hundreds of airports. What patterns can we identify among our datasets for pricing, weather, time of year, consumer behavior, plane model, etc.?
• Optimize existing models or processes
  We currently offer 2% cashback to our highest tier of customers, but what is the best balance of percentage and qualified customers? How would our profits change if we adjusted to 1.7% or 2.1%, or narrowed or expanded the qualified pool of customers? 

Data analysis and data science are complex fields with substantial overlap. Start by evaluating your end goal – what functionality or insight do you want to provide for your organization?  Next, consider your organization’s current level of data maturity and outline steps you will need to follow to meet your goal. Lastly, consider what skillsets will be needed to accomplish those steps and hire accordingly.