Hiring 1 Data Science unicorn is hard enough, a team is impossible. To scale means to specialise.

The Data Scientists need a large set of skills, including business know-how, modelling and mathematics, plus programming. They are as hard to find as unicorns, or superheroes. I know this talent shortage first hand. Is the solution to create more unicorns, or can we devise better solutions?

In my last role as a managing consultant in the Operations Research and Analytics team of a large global consultancy, I also ran recruitment. Having spoken to or met 150-200 of such candidates personally, and my recruitment team saw multiples of this number, I can tell you not many of those candidates made the cut. That's because they didn't have all of the skills we were looking for. And we were only looking for the first 2.5 of the 5 core skill-sets of a data scientist below. "Good luck" is what people offer to this talent-search problem, but I think we can get around the unicorns.

The 5 core skills of a Data Scientist 

Expanding on the data science venn diagram, I think the following 5 skills deserve closer attention, separately*.

• Business consulting (from problem definition to stakeholder and team management) :: what problem to solve
• Analysis and modelling (maths, stats, physics, OR, engineering, etc. / note this includes coding) :: how to solve it
• Communication and visualisation (artistic and functional, learn the visualisation tools) :: how to tell the story
• Data engineering (take data in, store it, push data out: computer science) :: how to get the data for the solution
• Programming (for enterprise use at production level, software engineering, integrating into BI systems, automated decision making embedded in operational systems) :: how to make the solution useful to a wider audience

Furthermore, each of the above have subfields and specialties, because they are complicated in their own right. It is not possible to be very good at so many things, not at scale anyway or to be above mediocracy at best. How many sportsman/woman excel at more than one sport, for example?

It's a lot to ask for one person. So, why ask just one person?

The thing is, these people all exist, have existed, and will exist. They are just separate individuals. They have labels like business analytics consultants, statisticians and modellers (operations researchers included), data visualisation experts, DBAs and software engineers. Yes, they are also talents in need, but they are not unicorns. If we need data scientists in troves, we need a team, not just a few geniuses.

The future I see is like the age old relationship advice: 

Don't try to change them. Instead, let's change how we work with them.

People should diversify a bit, for instance a modeller should be able to code, but ultimately they need to specialise in something they are good at. A modeller must be able to prototype on his/her own, which requires coding skills, but s/he shouldn't be expected to produce production-ready code for large scale applications. Similarly, asking a good modeller to do database administration and ETL tasks is a waste of talent, Hadoop or not.

Specialisation is the reason for humanity's proliferation. Therefore, I'd say it's not the people we need to change, but the system that we need to setup to allow such specialised workforce to team up together. It's lazy for the analytics field to put up its feet and just summon one person to provide it all.

As a starter-for-ten, I think the future of our field could be modelled after the traditional IT project group make-up:

• The technical "purists": analytics modeller, data engineer, visualiser, programmer
• The "bridge": more like a traditional business analyst
• The "glue": project manager with business consulting skills

There will be complications to address. To name a few...topics for another post:

• Who should start up your data science team?
• What's the load balance? (how much of each skill to have)
• How to coordinate the division of labour?
• Where should they sit in the organisation?
• How to prioritise the problems to set them working on?
• How to manage this team?
• What's the career path?

Where do you think the analytics field is heading to?


Disclaimer:
My views are definitely biased by my background: I am a manager in business analytics consulting, trained in Operations Research and Computer Science.

* My expansion on the data science venn diagram's 3 skills are based on various articles, such as O'Reilly guide, Intro to DS skills, and job requirements on numerous current data science job posts.

Even Google can't get their numbers straight

Google has so many various entities and products, either grown within the organisation or externally acquired. It appears that even Google, the leader in Data Science and Analytics, cannot get all the numbers straight across their products: Google Analytics vs. Blogger.

Is this blog really that popular? Really?

While I was checking this blog's traffic numbers on Blogger's built-in "Stats" function, I was really surprised that the blog seems to be really popular, even though I have not been good (sorry!) at writing much for some time. As an ex-SEO'er, I had an inkling that something is not right. Up comes Google Analytics.

Blogger Stats numbers are 4.5 times bigger than Google Analytics'.

After checking my Google Analytics (GA) numbers. I was really surprised to see that the Blogger Pageview numbers were 4.5 times bigger than the GA numbers. That is a staggering difference!

After some research on the web, I concluded that:

1. GA is much closer to the truth (but not quite completely true, see 3 below).
2. Blogger stats include all kinds of bots traffic, so it's heavily inflated (GA tries to filter most out).
3. GA cannot count any traffic if the user has disabled Javascript. Some folks suggest it undercounts traffic by 50%, but there is no hard evidence to back it up, so take it with a grain of salt.
4. Blogger seems set on reporting only Pageviews, not any other useful metrics, such as Visits or Unique Visitors. Not sure why.
5. This blog has probably been targeted by a spam bot. Upon closer look, one of the bots probably comes from a particular Dutch ISP.

Share best practice and be consistent.

I would have expected Google, the leader in Data Science and Analytics, to share best practice amongst its entities and products, such as reporting on key metrics (not just Pageviews).

I would also have expected Google to be able to have a consistent set of numbers amongst its entities and products. Doesn't appear so neither.

The majority of a Business Intelligence (BI) analyst's job is spent verifying and reconciling numbers amongst various reports, more often than not. Major BI tech giants sell BI applications that often allude to reducing such activities and increasing business confidence in the numbers in their data warehouse. However, it is still a major challenge to most companies, as evidenced here. Without a good and reliable data source, the validity of any following analysis is heavily undermined.

Let's try to stay consistent.
That goes for the metric choice, and the numbers.


FYI: if you want to find out if and who is attacking your site with spam bots, read this helpful post.

Coursera and the analytics talent gap

It's been a while, and ThinkOR is back to blogging about Operational Research and its related themes.

ThinkOR authors are about to start on 3 Coursera courses over the next couple months:

• Natural Language Processing
• Machine Learning
• Data Analysis (with R) (to refresh)

I am not only learning about some new topics for my own benefit, but also interested in assessing how such easily accessible courses could help the so-called 'big data and analytics talent gap' in businesses. As a Business Analytics consultant, this is one of the biggest issues I see my clients facing in today's business world - one wouldn't think about it, if they don't know about it, and once they know about it, they don't know how to get more of it. Obviously, there would need to be some sort of a step progression, such as (just an example without much research at this point):

1. Statistics One
2. Data Analysis (with R) and/or Computing for Data Analysis
3. some sort of programming course, check the computing course catalogue
4. Focus on one or several of the main OR techniques and their associated tools, such as Discrete Event Simulation, Monte Carlo Simulation, Optimisation, Forecasting, Machine Learning, and the good old Volumetric Modelling, as some examples
5. and if you are going to work with humongous data sets, Intro to Data Science sounds reasonable to become familiar with the various big data technology to apply data science (I suspect this often eludes traditional OR practitioners)

As ThinkOR goes along, we will be blogging about these courses and our learning experience. So far, there has only been very positive feedback. Let's get going!

Merry Christmas and Happy New Year!

Operational Research Consulting & Data Journalism

As data becomes more and more accessible, together with visualisation tools becoming more available and user friendly, Data Journalism is heating up. I've been following the Guardian's Data Blog enthusiastically, it is full of interesting information relevant to current affairs, explained with much facts and data.

This article talks about the 10 point guide to data journalism. I particularly like point 5:

Data journalism is 80% perspiration, 10% great idea, 10% output

The Prezi under point 5 explains the process of how data is used to support news, the angles to consider when mashing datasets together, the technical challenges of working with data, iterative calculation and QA process, which finally get turned into the beautiful output with the various (mostly free) visualisation tools.

This is practically the same process that an Operational Research consulting project takes - or any application of OR or Science in general:

• Understand what the problem/question is
• Create a hypothesis to be proven or disproved
  
• Define what data is needed for the quest
• Get the data
• Clean it, and manipulate/wrangle with it so it's usable for analysis
• Analyse/calculate to come to some conclusion - hence proving or disproving the hypothesis
  
• Compare it to subject matter experts' view on what the likely answer should be (sanity check)
  
• Refine the analysis until satisfied
• Shape the output message so it can be easily understood by the audience
• Communicate the findings
• All throughout the process, keep communicating to the audience to make sure they are engaged and understand (principle-wise) what you're trying to do, so that they are not unpleasantly surprised when the final answer is presented
• Best yet, to ensure smooth change management if your solution is to be implemented, work closely with the end users from the start of designing the solution, and then implement and test, so that they believe in the solution because they were part of the creation process.

As the Flowing Data blog points out, this is what statisticians do. I will add that this is what Science does in general. I will also say that in practice, the first step, "understanding what the problem/question is", often takes 70-80% of the time. The technical 'doing' to follow, in practice, is relatively easy compared to what our academic institutions thoroughly prepare us for (which is needed).

For those interested in the how of data journalism, read this about the work that went into reporting on the 2011 London Riots. Fascinating social media analytics at work. Not easy. Impressive and very interdisciplinary.

P.S. Most of this post has been sitting as draft since the summer, hence referencing 'old' news. It's still relevant, so why not.

What motivates us the most

First let me make clear that I am talking about the motivation in workplace. In personal life it's easy - in first half of our life it's the Sex, in second half it's the Comfort. (So to speak with tongue in cheek)

But the workplace motivation is more intriguing. And that is the area that every OR specialist should always keep in the forefront of their mind - the questions and aspects of human motivation. Here's an excellent animated video derived from the talk of one Dan Pink at RSA. Seems that Mr. Pink also excels in self-motivation, since this lecture is a small masterpiece.
True, these research findings are popping up here and there for the last two decades, at least, and lots of companies are adopting some of those principles, however this short video sums it up in excellent concise way. Enjoy!



However, I personally think that all these findings are missing some essential qualifications. I thinks that it reflects the motivation of people in developed countries, where there is no hunger and war is something nobody really remembers.
To echo the words of Mika Waltari in his book Egyptian Sinuhe, where he describes one lucky country he travels through, "...and the people who knew neither hunger no war, were already in middle age...".
I wonder, how the same research would turned out in war torn Angola, or Iraq.
I suspect that this type of "Make the world a better place" altruism grows best in economically nutritious Petri dish - relatively wealthy society. But what do I know about the poor countries. Maybe they would surprise us the most. The world is changing after all. It's the Internet age now.

One observation I made about the phenomenon of people working in their free time for free. (Linux developers, etc.) First I would liken it to simple hobby-ism. And I think that it indeed has the roots in hobbies. Everybody at some time in their life likes to build some "model airplane" and see it fly. But, and here comes my observation, they would like more to see it soar, than just fly. In other words, people don't mind to work for free on somebody's else project (i.e. Linux), but they prefer to jump on winning bandwagon. The likelihood of overall impact (let's even say world wide impact) is a specific motivation on its own.

It's the Internet age now.

Want to be creative? Don't brainstorm.

I'm sure many of us have had the thought before, "Oh, I wish I were more creative". I have. I'm also sure many, many of us have led or participated in a brainstorming session before. I also have. Apparently, these are both counter productive to being more creative, according to this article. To top it off, apparently since 1958 it has been proven that brainstorming doesn't work. I never knew about this. Did you?



In businesses, one of the common outcomes of operational research work is improving a particular process. We often start with understanding the problem, to mapping the process, and to building a model that reflects the current process. Eventually, to add value to the bottom-line, the model hopefully reveals some insights, and is the tool to test out certain ideas to support any process changes. Personally, it is often a pleasure to be involved from the beginning of problem understanding to the end, managing the recommended process changes, because as an OR consultant, you get to see your work to fruition.

Of course, to succeed in change management, the ideas should come from the stakeholders who live and breathe the process in question, and eventually own the solutions to be implemented. To get ideas from stakeholders for process improvement, the common technique is to gather stakeholders in a room and brainstorm on possible solutions.

Given the information presented in the article, to prepare for the brainstorming session, I take away that we should consider to:

• present the problem to the group before the brainstorming session,
  
• ask them to prepare and think about possible solutions that their colleagues or friends wouldn't have thought of for resolving the issues,
  
• get them back in a room to discuss each other's ideas and prioritise on the ones to investigate feasibility and impact,
• but before they start discussions, get them do some aerobics for 30 minutes if they are somewhat fit (half serious, but wouldn't that be fun?),
• culture them with a youtube video about the weird and cool stuff in other countries (half serious, but wouldn't that lighten up the mood?),
  
• facilitate the session with careful language to not instruct people to be creative
• facilitate the session so the group moves back and forth between a couple topics to be able to take a break from focussing on just one solution
  
• and perhaps not name it a brainstorming session, because it may be the forum that people associate with "get creative...now", which is counter productive, as per the article

Read it if you've got 3 minutes. Let me know what you take away from it that I've missed. (See the instant application?) The main points in the article to help someone to be more creative are:

• Don't tell them to be creative
• Get moving
• Take a break
• Reduce screen time
• Explore other cultures
• Follow a passion
• Ditch the suggestion box
  

Travel, being an OR consultant, and another blog

Activities on the ThinkOR blog has been a bit thin in the last month or so. Summer has arrived and we have been busy enjoying it as much as we can in London. So far it's been a great half year: Exeter UK, Istanbul, Bursa, Ayvalık, Bergama (Pergamon) TR, Riga LV, Berlin DE, Milan, Venice, Padua, Verona IT, the Algarve PT, New Delhi, Agra, Udaipur IN, Bahrain BH, Malaga ES, Reykjavik IS, and of course Canada and the US. Not bad, eh?

Travel:

To travel this much for leisure (18 countries last year), and to cover as many interesting cities as possible that span the continents, i.e. objectives; to not break the bank, to use as few vacation days as possible (we've only used 9 so far), to avoid anticipated bad weather, to not leave work too early for flights, and to not overdo it to tire ourselves out, i.e. constraints; means that we need an optimised strategy. We travel on weekends and use bank holidays as much as possible. We travel budgetly with lean (polite for 'cheap') airlines like EasyJet and Ryanair flying out after 5pm on a Friday, to trade off between more time in the destination and the cost of 1 extra night of hotel, as well as a peak rate for flights after 5pm. We make a judgment on the trade off between the central location of hotels with the higher cost usually associated. We also need to do our research on the temperature and the likelihood of rain for the cities on our list, and line the cities up with the weekends we would like to travel, but our list is often dictated and changed by the destinations of the airlines and the routes on sale. Our part time job is a travel agent, because it is quite time consuming. However, we usually plan a couple months in a batch process, and don't need to think about it again once it's in the diaries. It's kind of fun planning it, and more fun zipping away every second or third weekend.

Being an OR consultant

I just started a new job at Capgemini Consulting's operational research team. Already did one project with a major consumer product manufacturing and distribution company. Very interesting project, in which I enjoyed working on modelling their supply chain and the cash to cash cycle, and the impact of one seemingly simple decision's impact on the bottom line. This is exactly what OR is for - helping businesses make more informed decisions. The project was quite short and intense. I feel like one of the most important attributes OR people bring to the table in situations like this is what and when you can use averages, what assumptions are ok and what would come back and bite you in the butt. Perfection mostly takes second seat to delivery deadlines. It reminded me of what an advisor told me at uni, "what you learn at school will get applied very little in real life, because businesses never have the time to give to an OR guy to properly figure out the problems and solutions. They want quick answers and they want it now."

Another OR blog

Capgemini has a very cool group of OR people, and they have an OR blog too! Figure it Out. Check it out. Interesting articles on the real life applications of operational research, particularly relevant to UK topics. Of course, I will be writing for them too, as soon as I acclimate a little bit.

P.S. We at ThinkOR are very honoured to be named as one of the favourites in the OR blog world by Maximize Productivity with IE & OR Tools. Thank you very much. It is a real honour. Please let us know any topics you'd like to read about more, and we will try our best to research and write about them.

Healthcare system improvement project management: making a big team work

It's tough chairing meetings, tougher chairing a big meeting (10-15 people), and tougher yet chairing a big meeting that's supposed to last an 8-hour day, one day a week for 6 months. A lot of planning goes into making such a day work with team members varying from the analytical kind to the "feeling" kind, from the surgical kind to the managerial kind. I'm slowly to get a hang of it having done it for a couple months now. The following is a lot of common sense, but if one doesn't have the chance to go through this kind of work with big teams, one may not think it so obvious as an approach. Thought I'd share for whatever it's worth.

• Make sure everyone is doing something - feeling of usefulness in the group, or else people will feel disengaged.



• Assuming natural progress of project is from problem discovery, to analysis, to design and implement, and assuming that everyone in a team needs to participate in all phases, then keep telling self that as soon as we get through to design, things will become more exciting. Analysis phase is not everyone's cup of tea, even though geeks like me find it most interesting.



• Spend the time and create a big poster out of rolling parchment paper. It becomes a live document of all work done on the project to remind team in every meeting of key aims and work accomplished so far. It is a pat on the shoulder for work well done, as well as always showing the direction for the team. Sometimes, one can't see the forest for the trees.



• Big team, big scope - recipe for getting lost or losing sight easily; remind team of aims frequently; relate how current tasks contribute to the aims.



• Identify one lead for each main task to be done in the implementation phase. Give team members enough time to develop own plans on how to implement, and write the document themselves to instill ownership from the start (do not use admin resources to do this). Sometimes it takes 2-3 days just to write and re-write the implementation plans, but the time is worth while, not because we need to have a perfect plan as that is unrealistic, but because it forces people to think of all nitty gritties of how get things done and how they would get around specific change management problems. Provide a good example from a colleague of theirs (real examples from real people = trust), but encourage and give them room to be creative. Then everyone on the team should peer review each other's plan with specific review criteria.



• Once you have all of the above done, engagement level should be pretty high by now, as a healthy amount of sweat and tears will have gone into the implementation plans. I bet anything that you won't be able to hold people back on actioning out those implementation plans.
  

There you have a much happier and motivated team. There is no sure recipe. This isn't one by any means, but it is working for me so far.

Healthcare system improvement project management: how not to manage projects

Lately, I am finding it difficult to not do the work myself in the projects I'm leading/managing. The excuse I've been using is "well, it's just easier to do it myself than asking someone else for it". However, I end up paying for it with way too many late nights working around the clock. I'll be the first to admit: this is the wrong way to manage projects. I end up feeling burned out and tired doing work that should have been done by others in the team, leaving me without enough energy or time to actually 'manage' the projects. Ultimately, if I continued this way, it would be both bad for me and the projects.


However, I used to lead projects like this before, and it worked charmingly. What changed?

Well...

Here I talked about the Master of Management in Operations Research program that trained me as an OR professional (great program by the way). During this master's program, each student is a project lead on a 4-6 months project with a real company doing real projects. The students are fully capable of carrying out all tasks within the project, but have data analysts to help out, because there is just too much analysis work for one person usually. A project lead in this scenario is both the leader and largely the doer - what I'm used to do at work both before and after the master's program.

Why isn't it working now? In my humble opinion, leading 2 projects with relatively large project teams is quite a busy job. One simply doesn't have the time to both lead and do. I did, so I paid for it. Then I learn. I guess in this case, it would probably be overall easier to ask someone else to do it than doing it myself.

Got any tips to share with me? Comment here or email me at dawen [at] thinkor [dot] org

OR Career Path Talk by Jason Goto

Jason Goto came to the University of British Columbia and gave an informal talk & Q&A on career path in Operations Research as part of the INFORMS UBC Student Chapter event series. It was a very open dialogue appreciated by the audience. Here are some highlights from this talk.

Job Market:

Work opportunities in operations research locally in Vancouver is rather limited. It includes

• The big health authorities: Fraser Health Authorities, Coastal Health Authorities, BC Cancer Agency, etc.
• and maybe some engineering firms, such as Sandwell Engineering

Particularly, if an OR professional is looking for good job opportunities, one should consider relocation to the east coast or the States. However, we live in Vancouver because of a lifestyle choice, so if that's clear, prepare to sacrifice in pay and business opportunities. 

Jason stressed on the importance of critical mass of OR group to sustain an OR operation and presence within an organization. If an organization has only a few OR professionals working, they may not be able to achieve enough to show the importance of OR; and if someone leaves or goes on vacation, things grind to a halt and will take a long time to get back on track.

OR Consulting:

If an OR professional is thinking of going into consultancy by joining a consulting agency, then those companies will value the consulting/business/soft skills much more than they do about your technical OR skills. However, an OR professional should possess the following skill set:

• data skills
• consulting & communication skills (written & verbal)
• change management
• empathy - put yourself in other's shoes to help them understand your view

There are quite a few very small OR consulting companies with 1 to 3 people. Mostly they are academics doing consulting on the side. These small outfits don't tend to grow, because it is simply easier to do the work with only a few people, especially if you want the work to be done well, without much administration and supervision.

Jason's operations research consulting company, AnalysisWorks, incorporated in 2000, has been growing 25% a year. He has groomed it to an 8 person outfit - a steady, unaggressive growth since the start.

Starting a consulting company, the first year is the hardest. Everyone is against you if you have no credentials or portfolios to show. It is difficult to get the consulting projects because of that. Especially if you look too young (if you are starting out early), people don't take you seriously. You wonder if you are getting your market value. However, on the other hand, if you start the company when you are older, there are elements pressing against you as well: family, dog, house, pension, etc.  Some people may choose to start a company in groups. This requires careful consideration and an early agreement on who does what, just like in a marriage. If one partner is good at selling and the other good at doing, the two must agree on how they will operate together and the compensation scheme. Otherwise, break-ups could be very bad - again, just like marriages.

In general, it is difficult to get the OR consulting projects. If 10 companies are contacted, 1 may come back with some interest. Most people think the work is good, but do not think it absolutely necessary. The good clients are the ones that really, really need your help, because otherwise their jobs and the company's survival is on the line.

Over-delivery is a consultant's own loss. Clients may have been more than happy with the less than perfect solution, compared to a perfect solution which could have taken hours of the consultant's time - exhausting the budget that way.

Killer Presentation Skills

Steve Job like presentation skills are hard to pull off, unless you are a celebrated Toastmaster with a wealth of experience in presentation. However, here are some very helpful pointers to make a synchronized and seemingly effortless presentation!

• make your theme clear and consistent
• create a headline that sets the direction for your meeting
• provide the outline
• opens & close each section with clear transition
• make it easy for your listeners to follow your story
• demonstrate enthusiasm
• wow your audience
• sell an experience
• make numbers and stats meaningful
• analogies help connect the dots for your audience
• make it visual
• paint a simple picture that doesn't overwhelm
• give 'em a show
• identify your memorable moment and build up to it
• rehearse rehearse rehearse
• "And one last thing..." give your audience an added bonus to walk away with
• a strong opening, a strong closure, and an encore with "one more thing"

• 10 hours to rehearse for a new 30-minute presentation. It may sound like a lot, but if you want Jobs-style drama, you need to know your material cold.
• A Vision: If your topic can’t be summed up in 10 words or less, it’s too broad.
• A Clear Structure: An organized speech is easier for the audience to follow.
• Visuals: Eye-catching graphics form the basis of the most compelling slides.
• Dramatic Flair: A few time-tested storytelling devices help build excitement.

Click here to view the video.

Click here to see the full article.

Consulting & Communication: Are you told what you think you are told?

In business schools, the professors cannot stress less about communication in business consulting engagements. I whole-heartedly agreed with it before, and now I had the chance to experience it first handedly.

As an Operations Research professional or student, what comes to mind when someone says

• "we do cross docking"
• "our system is a pull system"
• "our system is 'just in time'"

Just when I thought, "wow, these guys are doing really well on their own", the answers I got after I asked them "what do you mean by..." the above statements usually left me with a surprised "Ooooh...THAT's what you meant!" - because the answers I got was completely different than what I understood were meant by those terms.

Communication - clarification - don't assume what you know, but always confirm what you think you know is true to make sure it is actually true - that is the lesson I learned.