creative technology – Blair Neal

May 20, 2024 — 2 Comments

Creative Technology Ecosystems

This is a sort of continuation on my essays on creative tech and organizational structures, but could easily be considered its own thing.

My previous essays on this topic focused more on projects and clients and their impact on the working process. This essay discusses the broader ecosystems connected to the tools used in creative technology and experience design, tools that the industry relies on to make their work. Understanding the origins of these tools can provide valuable insights for teams working on experience design and creative tech projects.

Consider the chart below which is a sort of high level map of how I think about the “sources” of the tools used by creative technologists.

When we compare creative technology and experience design to other industries, it becomes clear that it is relatively small. How many experience design companies focus on creative technology as a core competency? How many projects to they handle annually and what are their revenues and number of employees? Despite some projects gaining significant media attention, the financial impact of creative technology remains modest compared to other sectors. Consequently, many tools used in creative tech are adapted from technologies developed for much larger industries.

Why is this important? Consider a technology like the well known depth camera, the Microsoft Kinect. The Kinect is old and essentially deprecated, but stick with me. This tool was launched in late November 2010 for the Xbox 360, a gaming console for consumers. The tech has origins from Israel's PrimeSense, and then Microsoft acquired them and moved the tech towards gaming applications. This research and development took years, and likely hundreds of millions of dollars. The original Kinect sold very well to consumers (over 8 million units. Additionally, despite the fact that the tool wasn't explicitly made for the creative technology communities, the Kinect also eventually began to get a foothold as a new sensing technology for use in interactive digital experiences. It was everywhere for a while, and we can still feel echoes of the "wave your arms at a wall of particles" projects popularized by the Kinect even today.

Now let’s consider a hypothetical case of direct impact of the creative technology community actually buying that tech.

Let’s (generously) say there are 500 companies in the US doing creative technology projects in a given year. Of those companies, maybe Kinect projects are really hot right now, and they have 3 projects each that year that all use 5 Kinect's per project (so 7500 Kinects across all companies). At $200 a pop ($400 for the Azure Kinect), that brings the grand total of Microsoft's take to something like $1.5 Million for 7500 Kinects - an optimistic estimate and a slim percentage of the total 8 million units sold. While these projects might enhance the Kinect's visibility and perceived value, $1.5 Million is insignificant compared to the consumer market and other industries. On top of that, many of these large tech companies occasionally take on these projects not as a profit-making venture, but almost more as an intensive abstract marketing effort - make a "cool" technology that can be used in a really "cool" project and attach Microsoft, Intel, Google, or Apple onto it. Consumers may not go out and buy a Realsense camera, but they might remember that Intel does "cool" stuff like that. See also, the concept of a loss leader (the Kinect 2 was reportedly sold at a loss).

So if all of these technologies aren't being made for "us", the creative technologists - who are they for? Well, there is the consumer market, there is the defense and weapons industry (remember how Hololens got a $22B contract with the military?), there is the medical industry (see Google Glass Enterprise), and a lot of other high profile and well funded industries. And I'm mostly talking about the high profile sensor technologies that a lot of us use - there is a whole other side to this in large scale AV technology like LED walls, computers, microcontrollers, etc etc. Supply and demand still mostly rules here. Below is just a partial list of tools or frameworks and either their original “funder” or what I would assume is a primary industry for that piece of tech:

Defense/Military
- Depth Cameras
- Infrared and Heat Sensing
- Headset displays for VR and AR
- GPS
Retail:
- RFID/NFC
- Touchscreens
- "Digital Signage"
Automotive and Manufacturing
- Computer vision
- Various sensors (pressure, distance, etc)
- Robotics
- LIDAR
Entertainment (professional and consumer)
- Video and display standards
- Cameras, connectors and lenses
- Audio equipment and standards
- Displays, Projectors and LED
Medical
- Heart rate sensing

*Caveat that the above points are based on observations and not detailed research

Since these technologies aren't usually designed specifically for creative technologists, the discipline must get really good at adaptive reuse. A key skill for a generalist creative tech is to be able to see through the noise of the tech landscape and make recommendations about what will and won’t work - a lighthouse in the fog. We’re sifting through other industries to find tools that enable new creative capabilities and methods of expression. However, to my point above, creative tech can be a small voice to direct change in those technologies.

The idea of adaptive reuse gets at the heart as to why working on creative technology projects can feel either exciting or truly challenging. We're often using and bolting on tools that weren't designed for the purpose we're using them for. They are often tools that were profitable enough to be produced at a high volume for a broad set of purposes (or one specific case). This is certainly nothing new to the world of creativity or working in general, but it does answer some important questions that come up over and over.

Creative Tech people may know what we want or need in a sensor, or a depth camera (higher frame rate! high resolution!), a computer, a software tool, but there may never be something that gets released that is just the right fit - so we make it work. For people reading this that might be more non-technical, I think a key takeaway is allowing for some grace when things don’t work as intended.

So with all of the above and the challenges of a broader ecosystem, how can organizations be more responsive and less reactive to tech trends? What sorts of technologies can the creative tech space have greater influence over or is the only option to try and make your own product and hope you can recoup the costs? This topic is a meaty one meant for its own separate writeup, but at a high level I think that things like creating room and curiosity for prototypes and research both inside and outside of projects can be a huge help. Having an awareness and sensitivity to the forces at play outside of the creative technology and experience design space can be a crucial step to identifying paths forward.

May 8, 2024 — 2 Comments

Creative Technology and Organizational Structures – Part 3

Link to previous parts: Part 1 and Part 2

Project Structure within the Company

Let's zoom out further and look at the entire company and other elements involved in actual projects.

Here we're not only considering the departments and how they work together in a single company, but how the company works with the client and other vendors involved in bringing a project to life. Large projects inevitably involve large teams, and decisions made by one team can significantly impact others. The purpose of this section is to look at even larger systems at play, providing insight into the level of influence the Creative Technology team can wield in shaping the outcome of a project. Creative Technology is not merely an isolated group being passed tasks to execute; it is an integral part of a larger team of team, each driven by its own motivations.

Below is a partial list of some of those factors that might drive different teams but often can be diametrically opposed (see also: the good/fast/cheap triangle) :

User experience
- Ensuring users have a positive and meaningful interaction or experience.
User engagement/earned media/and “buzz”
- Maximizing reach, shares, and visibility of the project.
Representing “The Brand”
- Ensuring the final project aligns with the tone and quality expected by and of the client.
Creative output
- Striving for a high degree of polish and artistry, aiming for innovation.
Team makeup/staffing/morale
- Assessing capabilities and pushing skill boundaries of a team.
Use of technology
- Prioritizing seamless, engaging, and reliable technology.
Timeline
- “It’s Valentine’s day. This project needs to be live for SXSW in March”
Budget
- Managing financing constraints - often an immovable wall that influences most of the factors above

The push and pull between teams to optimize their goals above are what make projects challenging and fantastic at the same time. While all teams recognize the importance of these factors to some extent, their prioritization of those factors may vary. The tensions are also not often diametrically opposed foes - an increase in one doesn’t often equal a decrease in the other. A superior creative output can come for “free” and feel effortless with the right talent and vision. These can be contrasted with expensive, big budget projects that fail based on poorly conceptualized goals. A longer timeline and bigger budget doesn’t always mean the use of technology will be better or more magical for the user. It’s a complex, multi-dimensional problem, and I would be wary of a company that says they have it all figured out for every project. Internal cohesion and balance within a team are what help stabilize against the external pressures from clients, vendors, and market forces.

That said - I think it can be helpful to look at these forces not only within the company as in previous parts of this essay, but also in relation to external influences. In the next couple sections, I will show some charts that illustrate how a company and department might find itself within a project. These charts are drastically oversimplified models - we must acknowledge that each project is drastically different. However, they do offer some insight into the diverse decision-making processes within departments and companies, particularly in the context of short and long-term projects.

Agency and Production Company

To begin, we’ll look at a typical structure I’ve often encountered in fast-paced projects, typically associated with large or medium sized events such as booths at CES or pop-ups at SXSW (or something along those lines). It’s important to note that while this structure offers a glimpse into a potential approach, it’s not prescriptive of how things always happen - but just that they could.

In the first chart below, we focus on a scenario where a client or agency collaborates directly with the company or vendor organizing the event. That client may then hire or already work with an agency, and that agency would then hire a production company to execute the idea. For instance, you could frame this as: an automotive brand wants to showcase their new technology at CES using an interactive installation - they use their agency of record to strategize the installation, and the agency hires a production company to actually create it. The automotive brand is just a vendor at CES and is beholden to things like venue and event rules, but still has some sway - they could be considered a peer. Any vendor that the automative brand hires must also abide by these venue and event rules and restrictions ( for example: load in times, access times, power, internet, etc.)

It is worth noting that sometimes the event organizer or the venue itself will also have an established technology integrator that is responsible for installing all of the AV technology for an events space (or there may be a union involved). In those cases, the creative technology agency or production company will have to work directly with that specific AV integrator to make sure that everything is in place to make the event happen. Notably, the power dynamics are a little different if working as collaborative vendors versus a production company that hires and pays their own vendor directly.

Another aspect to consider regarding the chart above is the indicate layers of approval and acceptance within the “Client” category. While communication often occurs directly with client teams that may be involved primarily with marketing/strategy/creative, larger projects may require approvals from higher levels of executives that aren’t as close to the project. That decision making structure can have a lot of trickle down effects for all the parties below if big change or a wrench is thrown towards the end of the project that requires a lot of adjusting.

This communication dynamic is relevant for both event and “permanent” based work. It is crucial to consider how project status is communicated across various levels, the frequency of communication (and reviews), and who oversees the communication flow. Identifying who needs to be informed and involved at each stage is key to maintaining project balance.

Let’s consider a common path of: client makes a request -> sends it to account management -> account management asks production or tech if its possible -> production and tech teams report back on budget and timeline implications to address the change -> change is made (or not made). Sometimes a change is minor and considered within the realm of feasibility/teamwork/overall project vision, and sometimes it is major enough to trigger a considerable adjustment to the project (and budget). Sometimes a request is so obviously outlandish (“Can we have 10 of these instead of 1?”) that it can just stop before flying around to all parties, and sometimes it is less obvious and needs to be vetted (“What visual tweaks can we change on site?”).

Sometimes having more links in the chain (in the case above with the agency between production company and client) can sometimes complicate communication channels. However, experienced folks can recognize this and strive to communicate as clearly as possible so messages get passed along to the right people at the right time.

Next we’ll look at both a permanent installation/long timeline project model and one that has a client working directly with a company making the final product.

Direct to Client

Let’s explore another version of this for permanent installations and/or direct to client work. In this model, there is no intermediary agency between the production and the paying customer. To be clear, this removal of a middleman doesn’t mean that communication problems disappear; rather, certain communication challenges just morph into different considerations. This communication might be conceptually the same as the above, but that role of internal guidance of the client is just folded into the project and creative management on the production company’s side.

In this scenario, the agency doing the creative technology work is hired as a sort of "full service" shop, handling all of the aspects of bringing this project to life for the client. This can include project oversight, creative conceptualization, design, fabrication, development, installation, and maintenance, etc. For permanent installations, additional parties may be involved such as the client’s architect of record, the site’s general contractor, or even a content production team responsible for creating visual content.

This direct-to-client configuration is often preferred or seen more often in longer term projects which are typically more extensive (and expensive). Such project may span years and undergo team changes over time. Additionally, there is usually more time allotted for things like discovery phases, design, and production compared to event based projects. These extended phases allow the smaller team more time to deliberate on specific decisions and ensure they are the right ones before moving onto the next large phase.

Questions to consider at this level:

How do things change as projects scale up or down? How many cooks do you need to make a meal?
How many reviews or check-ins between teams are too many or too few?
What does the client's team look like? Do they have a project manager on their side helping to steer the ship or are there multiple stakeholders?
What is the value of an agency when working on large projects? There may always be a knowledge gap between the client's knowledge and interests and the expertise of the company doing the creative technology work. The layer of translation has to occur somewhere.
- Additionally, for very large branding campaigns that utilize things like print+websites+video content+social media AND experiences, it may not always be feasible or economical for the creative technology company to plan AND build all of that if they also want to focus on "just" creative technology.
How can you better facilitate communication between a client's marketing team and their technology team?
- Marketing teams can occasionally propose using their own product in a way that it is not designed to be used. The idea they have may not be vetted internally before beginning to work with an external creative technology team. Can you provide a common roadmap for the client on how to quickly arrive at a productive solution?
Is the value your organization brings to a project more about your creative strengths and ability to problem solve, or is it about some kind of secret sauce cutting edge technology that “only you” know how to wield? What are the costs of always staying at the cutting edge of technology versus working with reliable, tried and true methods?

That wraps up part 3 and generally all I will be covering on company and project structures. In the next part “Creative Technology Ecosystems”, we’ll kind of move on to something that could be it’s own series but in a way feels like part of the continuum of discussing the structures at play with this work. The intent in that section will be to look at where the technologies we use come from and how that can influence our working process.

April 26, 2024 — 4 Comments

Creative Technology and Organizational Structures – Intro and Part 1

Link to Part 2 Part 3

Experience Design and Creative Technology projects and companies can be an exhilarating mess sometimes.

This is not a problem unique to any one company - the “behind the scenes” processes of bringing incredible experience design work to life often feels frustrating to the people that do the work in companies large and small. Many projects have big creative visions, tight timelines, tighter budgets, move quickly, have vaguely defined boundaries, and are often working with cutting edge technology that inherently doesn’t have a clear path of efficiently working with it. No judgment - it is a really hard problem, and we could all use a bit of perspective sometimes.

In this multi-part series, I'm going to explore and reflect on different detail levels of the various organizational structures I've observed while working on professional creative technology and experience design projects. I believe that for everyone to do better work, Creative Technology needs more leaders thinking broadly and organizationally, not just on an executional “just make the thing” project level. These essays will be looking at this topic primarily from my personal perspective of working in creative technology in an advertising and experience design space, but I think there are some valuable insights for anyone working in the experience design field - producers, creatives, strategists, etc.

To be up front - I have struggled with the “who is the audience” for this piece, but wanted to finally share it out regardless. My hope is that students and newcomers can get a peek inside the industry and see where they might want to land, and department leads can get some perspective on how different companies approach the issue of process and structure, project managers can better understand the nuances of creative tech resourcing (i.e. freelancers vs staff, varied irregular skill sets). I think that by considering the role that you and creative technology play within your organization, you can start to think more operationally about how to improve your process and work more effectively with other teams and clients.

Overview of Upcoming Essay parts:

Interrogating the systems involved at multiple levels of detail can (hopefully) help us improve the ways that we work together, but this multi-level view can also give us a way to grapple with some of the real challenges of this kind of work.

In Part 1 (this part), I’ll actually start at the company level to provide some broader context first.

In Part 2, I’ll be exploring the micro level of just a hypothetical Creative Technology department within a company

Part 3 zooms out to the project level to bring it all together. We’ll look at the company within a project and compare direct-to-client versus an agency experience. It will make sense in the end, I promise.

For Part 4, I'll cover my perspective of how other industries "feed" the creative technology space and ecosystem - i.e. where do new technologies originate from and how do they become common tools for use in various creative technology applications?

In Part 5 (also at a later date), we'll actually turn inward a bit and cover some more philosophical points about how someone's personal (or a company’s) hierarchy of values will influence the work that is being created.

I think the value of looking at this field from these different levels is that it can help give some perspective from the day to day noise, and allow us to think about how to bring more meaning to what we do and how we collaborate.

Preamble: What do you mean by Creative Technology?

“Creative Technologist” is still a job title that many find hard to describe. The title ends up meaning very different things at different companies, and it can be hard to pin down what creative technologists could and should be doing within an organization.

For newcomers to creative technology, when I'm (personally) speaking about creative technology I mean projects that involve a wide range of things like: data-driven generative visuals on large LED walls, cameras and AI, buzzy buzzword technology activations, tried and true thoughtful and artful tech, interactive installations with sensors and robots, experiences that incorporate technology in creative ways, etc. There is a wide spectrum of work from pre-rendered assets to generative real time systems, and physical interactions to web-based interactions. The projects come to life at events, online, as mobile apps, as physical installations at museums, corporate lobbies, public spaces, and many other places.

When attempting to explain to my parents or other folks what I do as a Creative Technologist, I recently find myself reaching for this explanation:

I am an artist, and just like a painter’s medium is the full range of paint types and canvases, my medium is technology itself. I am considering the full offerings of hardware and software - their strengths, their weaknesses, their user experience, their reliability and many other factors. I look at the strategy, the concept, the timeline, the team, the landscape of technology and make informed choices about what technologies could be utilized to make the most enjoyable and thoughtful experience for a user. I also make and implement those choices to bring an actual project to life. In my personal experience, so much of what I do is about the theater of experience and picking and choosing technologies to bring that vision to life in thoughtful ways that honor the person experiencing something and the team creating it. Keeping things simple and letting the technology support the concept are some guiding principles.

For more overall context on what creative technology encompasses, check out this taxonomy I created and my advice for creative technologists.

Part 1: Structure Overview

Company Structure

Let’s first take a look at the broad range of organizations that may have creative technology as part of their offering. There can be a range of things to consider here, like degree of freedom in the work and whether you have clients or no clients at all. Depending on the size and "flavor" of your given organization's focus, your overall structure may be radically different. First - let's consider the range of organizations that creative technologists often find themselves in:

Solo Artist or Freelance Creative Technologist
- Doing your own work or contracting with a company. Nobody can tie you down.
Artist's Studio
- Working for a big name artist or a person who has a particular way of working
Startup
- Maybe an experience design company, or maybe more product focused (or both)
Agency
- Advertising or otherwise. This is a fairly fluid category where the ratio of “idea making” to “real making” can vary wildly within each agency. Some agencies just make ideas and hire talented partners to help bring them to life, and some are full-service and do everything from strategy, concept to execution.
Firm
- Architectural, Design Firm, Design Consultancy, etc. Some minor differences from an agency, but most of that can perhaps be a semantic exercise.
Studio
- Fine line between what you would consider a studio versus an agency - perhaps these do more work for cultural institutions
Production Company
- Handling many aspects of a complete production, but often are closer to execution of the idea than the original concept creation of that idea.
Brand
- Brands sometimes have their own "innovation labs" that do creative tech R&D
Product
- Some startups or products start in a sort of “Design Innovation” stage and need creative technology to bring it to life
Content Studio
- Primarily concerned with making digital assets like motion graphics, but many of these are beginning to bleed into experience and have their own creative technology departments
Live Visuals and technology for Touring Music Acts/Theater Productions
- Primarily things that fall outside of the scope of typical lighting/sound tech
Many others...

Personally, I primarily have experience with the agency model, but most companies have somewhat similar structures and types of departments. The differences seem to mostly be in labels, and the overall scale of the other departments. There are also often echoes of whether a company started off as something else and then eventually integrated or bolted on a creative technology department. For example, a company that was originally a motion graphics company may have a lot more motion graphics artists, art directors, and creatives than a company that was originally more of an architecture firm that may not have as much content creation as a core offering.

Starting from the top, a CEO/Founder or a few founders will typically be the key decision makers. Next up, an Operations and/or Production department handles everything that makes the business run smoothly - financials, project management, process, hiring, contracts and legal, etc. The Creative team handles everything from concepts to copywriting, art direction, and asset creation. The Technology team may be a mix of engineers and more "Creative Technologists." There is also often a Business Development team that helps to bring in new work and grow existing relationships. Strategy is a mixed bag of whether it exists as its own discipline at different agencies depending on the size - some creatives are expected to think like strategists, but it really is its own speciality that works to clearly connect creative work to business goals.

There are lots of different ways to carve up these departments and their purpose, and there are certainly other departments that could be added to or divided in different ways - architectural, PR, design, UX, systems, etc. The important thing to note here is that each department typically has their own goals and levels of success measurement that (hopefully) ladder up to some larger company goals and values. Creative Technology departments don't succeed all on their own, they need some mix of help from many other specialties.

Also - as you can see above, there are some big hypothetical differences when scaling up from just a few people to a much larger team. Some companies may have a totally lopsided structure that indexes much more in one area and farms the rest out to other companies and trusted partners, etc. and the diagrams above are really meant more to illustrate potential structures at play.

Finally, I'll end part 1 on some provocations and questions that I feel come up a lot at this sort of level. While I could provide my own answers and insights for each of these, I feel like it would make this exercise overly long - but perhaps one day I'll do a breakout of my own thoughts on each of these.

Questions for Department Leads to consider at the Company level:

How can the Creative Technology team work most effectively with other departments?
- Can budget and timing estimates get more accurate for the production team?
- Can workflows and technical hurdles be more clearly explained to the creative teams so that there are less misunderstandings as the project starts to take shape?
- Can concept prototypes be made more quickly so the business development team can sell through an idea to a client?
- Is it worth it to do a risk assessment of a project and consider all unknowns and points of failure ahead of starting so that all teams are on the same page about where things may need to pivot later on?
How can other departments work most effectively with the creative technology team?
- What boundaries are necessary around a typical development cycle and result in the “best” product?
- If a waterfall development method isn't yielding appropriate results, how can all departments adjust their approach to allow more time for testing and iteration?
- How can other teams work more collaboratively to understand the challenges facing certain technology solutions and work together to find an ideal solution?
Are departments more effective when they "stay in their lane" and are siloed, or when they are cross-functional and interdisciplinary? What is the line of crossover?
What is the role of research and development work for the creative tech team and how should that work be disseminated amongst the other teams?
Why don't companies more openly share things like org charts and the way they work (either internally or externally)? Is it a sign of a secret sauce, a lack of structure, or just that there may be limited value to others?
Is the creative tech team a "core" offering for the company that the rest of the company is truly investing in, or does it seem like more of a curiosity with a short shelf life for the company?
What parts of producing a creative technology project are unique? What additional skills would you want a project manager or producer to have if they are coming to join the team from an events background/software background/content production background?
- How can you effectively add in a "typical" software development cycle approach to a shop/company/team that is used to working in a different way? Does the software process need to adjust more than the rest? Where can everyone compromise?

Link to Next Part: Part 2

In part 2, we will cover more of the details around an actual creative technology department within a company. And in Part 3 we'll look at both that department and the company as a whole within the larger ecosystem of a company. Links will update here in the coming weeks.

Acknowledgements

I've actually been sitting on this set of essays for a little over 2 years, and I've been sharing it privately over the course of those few years and soliciting encouragement and feedback. I've gotten a ton of useful notes from a number of peers and friends (some of whom may not even remember helping out!) that I wanted to acknowledge sooner than later since it may take some time to publish the last few parts.

November 13, 2023 — No Comments

Creative Tech, Art, and the Afterlife

A brief workbook on how you might want your digital artworks preserved when you pass on.

October 19, 2023 — No Comments

Survey of Alternative Displays – 2022 Edition

You can find my Survey of Alternative Displays on gitbook here

Github link for contributing is here

As of October 2023, I also now have a PDF version of all of the above that is downloadable here.

The PDF formatting is a little ugly due to the export process from a pile of markdown files, but it is at least one single document that could potentially be printed.

2023 or 2024 updates will be coming soon after! Please send me any suggestions you may have for additional things to add!

December 30, 2021 — 4 Comments

A Creative Technology Taxonomy

⭐️ New 2024 interactive version of the taxonomy is here ⭐️

Link to the less compressed full-size image above

Creative technology is a topic that that I've been focused on for years. I've been writing some deep dive pieces on various topics like display technology, installation maintenance, and projectors, but there has always felt like more to cover. There has always been a feeling of wanting to tackle something even larger, like a book, but articulating the reasoning for going after something larger eventually feels like a good first step.

Creative technology is a discipline that has been evolving from other fields for a long time, but every year it feels like it is becoming more and more of an established path in academia and professional circles. However, I feel that there is a still a lack of clarity about what this field is and what it encompasses. Creative Technology is a discipline that is exceedingly broad, and most people and organizations operate within a niche of the overall field. Developing a common language and classification system for creative technology and a set of grammars to work with can really allow us to make better "music" together and help educate newcomers in a broad, but quickly digestible way. Having a taxonomy is also a useful way to have a historical mile marker for the state of the discipline in 2022, and maybe have a way to see what has evolved and changed 5 or 10 years from now.

To continue this conversation, I'm supplying a visual map of various tools and concepts utilized in the creative technology space. I originally made a version of this back in 2018, but have been updating it since then. As a caveat, the diagram is flawed as a hierarchical tree diagram and could really use an alternate method of visualization because of how complex and interconnected many topics are. To help with expanding this idea in the future, I'm also releasing a JSON file on Github that represents the exact same structure in my (originally) manually made visual map. The hope is that this file can be used to create alternative visualizations of the same data on an interactive site, and perhaps eventually be translated to other languages. One thing to keep in mind is that this classification system is primarily from my narrow perspective, and most of the map represents my personal awareness of certain areas.

Creative Tech Taxonomy PDF (8161 downloads )

Creative Technology Taxonomy 1.4 PNG (7656 downloads )

Why make all this?

A large part of my background is in music, and that knowledge base often feels like it influences this approach I'm trying to articulate. As an oversimplification, music theory is a set of mathematical and conceptual systems that facilitate the creation of music between people. It is keys, modes, notation, rules of harmony, chords, instrumentation and many other ways of describing everything from silence to noise. You don't need to know music theory to make beautiful music, but formalizing the systems behind music has unlocked a lot of different areas of potential. I would argue that the primary function of music theory is for communication - for example, allowing people to play in the same key when improvising, but also to talk to the past and read and play some sheet music composed hundreds of years ago. Music theory gives beginners a place to explore and understand in a known, quantifiable space with seasoned professionals. It is a system that enables creativity.

Aside from looking at the known - another huge function of music theory is that it gives people a language and a set of tools to also explore the unknown in a systematic way. Learn the rules so you can break them in a meaningful way. The constraints should feel more freeing than limiting. Like Schoenberg's twelve tone row, people may look at the idea of standard musical keys, throw that out entirely, but keep the other formalized elements of rhythm and notation to develop something new. They can look at the very concept of tonality and find a space between that and noise that somehow still speaks to us in a deep, ineffable way. It is possible we would have stumbled on some of these musical discoveries by accident (and some we absolutely did, don't get me wrong), but I would argue that forging into the auditory wilderness for exciting discoveries was greatly helped by a collective system that helped people understand why you went there and also how to get there themselves. New edges are found all the time that continue to push music into exciting spaces year after year.

Instruments could have remained a group of individual noise makers that worked on their own (and probably worked a bit like that for a while), but as soon as people tried to get one to talk to the other in a pleasant way, the effect snowballed into some of the most incredible forms of human expression and art the world has ever seen. Strings, woodwinds, brass, percussion and everything else - composers know all the options and what their strengths and weaknesses are.

All of the above is to say: developing taxonomies and defining systems are critical elements of getting to better art. People can have a common language, a way to describe what works and what doesn't. They can define trajectories over past decades and look at where things are going in the future. Taking a big picture view of the creative technology space can help us see the forest for the trees as well.

So what kinds of systems and taxonomies am I talking about, specifically?

As I mentioned, I've written things like guides to: cameras, projectors, alternative displays - all with a focus geared toward making interactive installations. I did this because I couldn't find anything comprehensive at the time. When coming up with creative solutions for creative technology projects, it sometimes felt like I was starting an undefined research project every time. Most projects had a need of an input method, a processing method, and a display of some sort for feedback to a user. If all you know of displays are LCD monitors and touchscreens, that might be all you might think of for output. If all you know are webcams for interaction, that's all you'll ever use.

Of course there is other stuff out there, but not everyone has the time or interest to do the research - and frankly, they shouldn't have to. Research into this stuff is important, and can help with learning and finding solutions quite a bit. However, the process of deep thoughtful research can often feel divorced from the process of free flowing creativity. Having a shorthand reference of tools, approaches, and rules can capture sparks of creativity that wouldn't have been there otherwise. Imagine if musicians had to agree on a new tuning system or had to make their own instruments every time they wanted to get together and play.

Defining creative technology systems is also a critical task for inviting newcomers.

The systems are here for you, come play music with us.

The thoughtful creativity you bring to the table is considerably more important than knowing how to wield all levels of a complex technology stack. Personally, open source software tools taught me so much about this. A small group of individuals working on tools like Processing, Openframeworks, or Cinder abstracted the approach of difficult low level programming problems and saved literal years of development for everyone who used those tools after them. Going through and learning the end-to-end pipeline for drawing a pixel on screen from scratch is certainly a worthwhile effort for some, but others just want to draw a circle on a screen and make some art. I'm speaking about more than just open source software here, though. This is about defining some of the other layers to the puzzle of finding solutions with creative technology - like hardware, displays, sensors, testing, etc.

On the professional and industry side of things (at least in the West), there is some protectiveness to sharing some of this knowledge that ultimately impedes creativity and innovation. Individuals at companies spend a lot of time solving and researching the same problems over and over, typically low hanging fruit problems that have already been solved by other companies that chose not to or didn't think to the share the results. Content management systems, people tracking with cameras, stretching images across multiple displays, uptime monitoring, etc. - most could be solved once and distributed. Available technologies and solutions are largely the same for everyone in companies that utilize creative technology, but their process and creative approach for how they used that technology should really be what defines the company's value. There are absolutely companies that share a lot of tools and findings, but hey - there should always be more.

To close, my taxonomy visual is not meant to be definitive or even remotely correct, just presented as a conversation starter. With every new branch I added, I questioned the utility of what I was adding and thought about how other people would classify things in completely different ways. I also think its only one piece in the puzzle towards creating a common language and way of working. Music theory is not just a list of instruments, but a whole language of collaborative expression. I would love to join with others on a way to keep this open and expansive - I think setting up a GitHub pages link with a graph driven the taxonomy JSON would be a great next step.

Thank you for reading. I'll close by sharing some links to some other great resources that help to cover a wide range of creative technology topics. Please leave any comments below, or send me suggestions (or a pull request) about what to add to the map.

New Interactive graph here

Terkel Gjervig's amazing resource covering many topics: https://github.com/terkelg/awesome-creative-coding

John Mars' resource for companys and organizations: https://github.com/j0hnm4r5/awesome-creative-technology

30+ Dumb / Creative Ways to use Apple’s AirTags

I think that the new Apple AirTag is a deceptively revolutionary piece of tech, especially for the experience design and creative technology world. Here are 30 creative ways they could be used.