Effective Problem-solving and Creativity's Role in All This

The problem with problems these days

There have always been problems we have had to solve across various domains. So, how are today's problems different? Are we getting softer of something? Or is it that these problems are somewhat harder and involve more complex parameters?

As more and more data becomes available and is factored in our decisions, be it on a smaller or a larger scale, we encounter problems that are riddled with the complexity this data entails. In the past, we could often work out the solutions with some math formulas and a good calculator. Maybe if we wanted to show off a bit, we could even use a spreadsheet program. But nowadays, none of these tools could cut it unless the problem at hand was greatly simplified or we were tackling a high-level problem involving aggregate data.

There is even a meme about modern problems which has entered the zeitgeist with its wit and relevance: modern problems require modern solutions. But how accessible are those solutions? Is it just a matter of using the latest software, or does it take something more? Let’s endeavor to answer these questions by first examining problem-solving as a skill, how creativity comes into play, and what problem-solving heuristics can do to help us deal with this meta-problem of complex problems.

Problem-solving as an essential skill

We cannot get rid of problems, or make sure they don't come about in our work. However, we can train ourselves to be more effective at tackling them. Problem-solving is often viewed as an essential skill today, in many data-related roles, and its popularity has only increased over the past years. With knowledge becoming increasingly accessible and immense, it ceased being a differentiator between the good and not-so-good information workers as it used to be. Nowadays, you can go to a series of websites and get all the knowledge needed to tackle a particular challenge, but this can only get you so far. When problems arise, you are on your own, so to speak.

Will you be able to break down the problem into manageable pieces and solve each one of them? Will you be able to come up with a method for solving similar problems proactively? How will you communicate your thought process and the solution to the project stakeholders? These are all essential qualities expected in all sorts of professionals today, from the entry-level analyst to the program manager. To make the recruiting process easier, these qualities are often grouped with the term "problem-solving" and it's come to be an essential skill in all these professions. This is particularly the case for more hands-on roles where you deal with data daily.

Creativity as a soft skill

When someone glances over particular role specs, creativity sometimes comes into play. This is observed not just for inherently creative roles (the creatives) but also for more conventional roles, where you wouldn't normally expect that quality to be required. In data science, this is often the case since you need to be creative when you go about your projects as they often involve unforeseen challenges.

Creativity is not all about coming up with new designs and such, though that may be the case too. This valuable soft skill is often associated with a different way of looking at things so it's tied to perception. Also, the idea that it is only characteristic of a set of "gifted" individuals is gradually dying since more and more people learn to be creative in one way or another.

Creative problem-solving

The place where creativity shines these days is problem-solving. How often do we hear things like "creative solution" and "creative approach" when dealing with a particularly challenging problem? Perhaps there is something to creativity that makes it applicable to problem-solving too, especially today.

Creativity thrives in situations where there is lots of space to explore. Perhaps that's why it's so popular in places like art and such. In the STEM world, there are similar scenarios too, like the complex problems we mentioned previously. This brings about creative problem-solving, essentially the application of creativity in problem-solving, particularly wherever problems of high complexity are involved.

Heuristics as problem-solving tools

What about heuristics, though? What are they, and how are they relevant to this creative problem-solving business? In a nutshell, heuristics are specialized tools designed to help us handle complex situations (usually problems) and derive a "good enough" solution for them. This is often more than enough for coming up with a practical solution to the problem at hand since the extreme precision a mathematical process offers is often unnecessary, and we end up rounding this up at the end anyway.

Heuristics are a powerful approach to problem-solving as they often involve some creativity, particularly if they are new and designed with that particular problem in mind. Usually, they aren't just mathematical constructs though but also efficient implementations of them in a programming language. Whether we use them for measuring something, providing insight regarding the data, or coming up with a good enough solution to an optimization problem (the most well-known scenario of heuristics) without using too many computational resources, they add value to a data project.

Naturally, there is a limit to what heuristics can do as there are no panaceas in the world of data. However, if you are aware of that fact and do not let them become a replacement for your creativity, they can be invaluable. And if you have a knack for problem-solving programmatically, they can be a great way to tackle even the most challenging of problems.

Final thoughts

One great way to learn more about this fascinating and practical topic is through my latest book, The Data Path Less Traveled. There, I offer a distillation of my experience with heuristics, coupled with plenty of hands-on examples from common data-related tasks. There is even a section for developing new heuristics to creatively solve problems and offer additional insights based on the data at hand. You can learn more about this book at https://wakelet.com/wake/aXyH6AAmst9dnvH8vvQA9. Cheers.