What makes a great manager isn’t the problems they solve, but the questions they ask. Start with these 16 questions here.
- An employee comes to you and says, “I have a problem.” If you’re trying to be a great manager, what do you do?
Your initial instinct might be to roll up your sleeves. “Time to be the boss,” you think to yourself. You’re ready to step in, solve the problem and save the day.
Or something like that. You just want to be helpful.
In reality, your instinct is the opposite of helpful. Startlingly, when you jump in to solve a problem as a manager, it’s one of the biggest leadership mistakes you can make.
I was reminded of this counterintuitive concept when chatting with Wade Foster, CEO of Zapier, on our Heartbeat podcast. Though his company today is thriving with over 200 employees and over 2 million users, Wade admitted how he struggled in the early days as a CEO when an employee would come to him with a problem:
> “When you [jump in and try to solve the problem yourself] you’re actually mistaking your roles. You’ve hired this person to solve problems. And if they’re unable to solve the problem, you’ve probably hired the wrong person.”
In other words, your role as a manager is not to solve problems. It’s to help others solve problems, themselves. Leadership is stewardship. It’s navigating your team through treacherous waters, around jagged rocks, to the desired destination, and making sure folks feel nourished and rested along the way. But you can’t be a good steward if you’re scampering around trying to paddle all the oars faster, yourself. To take the boat analogy one step further, a great manager is a coxswain, not a rower.
This confusion of roles leads to a highly undesired outcome: You prevent your team from learning how to solve the problem. A dangerous reliance develops that hinges on your expertise, your “final word.” Your team never gets to fuss, flail, and figure out how to crack a nut with their own hands. When you’re the one thinking through all the problems, you’re teaching your team members to not think for themselves.
You also inadvertently slow your team down. Every problem – especially the “hard ones” – are re-routed to you. So what happens if you’re out of the office that week? Or, what if your plate is full? Well, that problem will just have to wait. And wait it does. You become a bottleneck, the inhibitor of your team. You funnel your team into single mode of dependency that’s difficult to undo.
The best leaders know this, and are keen to avoid this pitfall – so they do something else. They become the team’s accelerator. They help team members think for themselves.
How? By asking questions. Wade of Zapier adopted this practice as a CEO, describing it as a “more Socratic way” to helping his team solve problems. Ultimately, it leads to better results.
Ask questions and a team member can come to the answer themselves. Ask questions and the problem they’re facing becomes more lucid, less daunting. Ask questions and your team member might even come up with a better answer than you would have.
To be a great manager, here are 16 questions you can start with instead of jumping in to solve the problem yourself:
- What do you see as the underlying root cause of the problem?
- What are the options, potential solutions, and courses of action you’re considering?
- What are the advantages and disadvantages to each course of action?
- How would you define success in this scenario?
- How do you know you will have been successful?
- What would the worst possible case outcome be?
- What’s the most likely outcome?
- Which part of the issue or scenario seems most uncertain, befuddling, and difficult to predict?
- What have you already tried?
- What is your initial inclination for the path you should take?
- Is there another solution that isn’t immediately apparent?
- What’s at stake here, in this decision?
- Is there an easier way to do what you suggested?
- What would happen if you didn’t do anything at all?
- Is this an either/or choice, or is there something you’re missing?
- Is there anything you might be explaining away too quickly?
What you’ll notice when you ask these questions is that most employees already have an answer (or several answers!) to a given problem. But they were uncomfortable with it, or they were worried about getting it “wrong.”
Part of asking the questions isn’t just to help them think through the problem more clearly, but also to help them realize they know more than they think, they’re more capable than they think, and that they’ve mitigated the risks better than anticipated.
Your job as a leader isn’t to just help clarify thought process – but to give confidence in their thinking.
As Wade says, “You’re trying to just help them get to that realization that, ‘You know what to do.'”
After all, a great manager is centered on building the capabilities of their team, not their own capabilities
Don’t solve the problem, yourself.
Claire is the CEO of Know Your Team – software that helps you become a better manager. Her company was spun-out of Basecamp back in 2014. If you were interested, you can read more of Claire’s writing on leadership on the Know Your Team blog.
- SuicaEng App
- Add Suica
- Add a balance
Finally! After 2 years of waiting for a full blown English version of the Suica App with all the bells and whistles, JR East has done the smart thing and released the SuicaEng app instead. This simple streamlined English app does one thing: add a virtual Suica card to Apple Pay without a Mobile Suica account or any of the hassle of dealing with the Japanese only Suica App options. It does nothing else but should take care of most immediate inbound needs. The same virtual Suica restrictions of Suica App apply:
- Only one Suica can be added, if you already have a Suica in Wallet you cannot add another one with SuicaEng, a second virtual Suica requires the full Suica App and free registration of a Mobile Suica account.
- Once added, Suica is managed in Wallet or Watch app.
The best thing would be adding virtual Suica directly in Wallet without the need for an app, perhaps Apple and JR East will deliver that eventually. Meanwhile anything more complicated than adding virtual Suica: purchasing e-tickets, commuter pass, Green Seat reservations etc., still requires the Japanese only Suica App.
I still need the Japanese one, but this is huge for me!
A passenger sneaked a firearm through airport security in Atlanta earlier this month before flying with it to Tokyo. This has attracted a lot of media attention, with CNN, Time, CBS, The Hill, The Washington Post, and others publishing write-ups of the incident.
So is the shutdown making airports less safe? Was it the stalemate in Washington, D.C., that allowed someone to slip a gun past TSA screeners?
The short answer: probably not. The story about the firearm appears to have been first reported by WSB-TV, an ABC affiliate based in the Atlanta area. On January 2, a man boarded his Delta flight to Japan with a firearm. Once he landed, he informed Delta workers that he had a gun. Delta in turn informed the TSA, who said in a statement that “standard procedures were not followed.”
The TSA insists the shutdown had nothing to do with the incident. “The perception that this might have occurred as a result of the partial government shutdown would be false,” the agency said in a statement to the press. “In fact, the national callout percentages were exactly the same for Wed, 1/2/19 and Wed, 1/3/18 (when there was no shutdown)–5%,” an agency spokesperson added in an email to Reason.
In other words, this wasn’t the shutdown; it was just normal TSA incompetence.
Sounds plausible to me. The TSA has a pretty bad track record when it comes to identifying items that could actually pose a threat. A 2015 Department of Homeland Security (DHS) investigation, for instance, revealed that in 67 out of 70 cases, undercover investigators succeeded in smuggling weapons or explosives through security.
Security theater, indeed.
“Everything changes and nothing stands still.”
As quoted by Plato in Cratylus, 402a
“It is harder to fight against pleasure than against anger.”
As quoted by Aristotle in Nicomachean Ethics, Book II (1105a)
“Time is a game played beautifully by children.”
As quoted in Fragments (2001) translated by Brooks Haxton
“War is the father and king of all, and has produced some as gods and some as men, and has made some slaves and some free.”
Hippolytus, Ref. haer. ix. 9 (Fragment 53). G. T. W. Patrick, 1889
“Though wisdom is common, yet the many live as if they had a wisdom of their own.”
Fragment 2, as quoted in Against the Mathematicians by Sextus Empiricus
“Much learning does not teach understanding.”
“The road up and the road down is one and the same.”
“You cannot step twice into the same rivers.”
Fragment 91. Plutarch, On the EI at Delphi
“Dogs, also, bark at what they do not know.”_
“It is better to conceal ignorance than to expose it.”
“Character is destiny.”
Paying attention to cybersecurity is more important than ever in 2019. But, some companies are still unwilling to devote the necessary resources to securing their infrastructures against cyberattacks, and naive individuals think they’re immune to the tactics of cybercriminals, too.
For people who still need some convincing that cybersecurity is an essential point of focus, here are six reasons.
1. The Average Cost of a Cyberattack Exceeds $1 Million
It’s no surprise that cyberattacks are costly, but some people will likely be shocked at the massive expenses that could result. According to a recent report from Radware, the total costs are more than $1 million. Additionally, victims report issues not directly related to financial losses, such as decreases in productivity or negative customer experiences.
Based on the above statistic, enterprises should conclude that although it costs money to invest in cybersecurity strategies, the expenses could be more substantial if organizations choose not to put enough of their resources toward experts and tools that minimize threats.
2. The U.S. Government Says It’s Time to Come Up With a Better Plan
The U.S. government, as well as the authorities from other nations, continually struggle to safeguard against digital attacks from rivals. The challenges are so immense that government bodies and officials warn that the United States needs an improved way to stop adversaries.
A State Department report warned that the country is increasingly dependent on networked information systems, and foes from other nations have learned to exploit that dependence and use it to disrupt the lives of Americans.
Most people who live in the U.S. can at least imagine the consequences of a severe cyber attack that affected the country’s ability to proceed with normal operations. Since government authorities researched the possibility and asserted there’s no time to waste in coming up with an improved approach to cybersecurity, that’s all the more reason to take action this year.
3. The Methods of Attack Are Diversifying
A decade or so ago, people typically felt sufficiently secure online by installing anti-virus software on their computers. That’s still a worthy precaution to take, but it’s no longer adequate for preventing all or even most of the attacks a hacker might try.
According to a 2014 report, cybercriminals orchestrated 75 percent of attacks through publicly known software vulnerabilities. But, they also try to gain people’s credentials through phishing attacks, lock down their systems with ransomware or infiltrate poorly secured connected devices to name but a few possibilities.
People have a growing number of ways to use technology and rely on connected devices, but that also means the likelihood goes up for potentially unfamiliar kinds of attacks. Focusing on cybersecurity this year requires, in part, understanding the most recent and common types of threats and protecting networks against them.
4. Recent Breaches Victimized Millions
Equifax and Starwood/Marriott dealt with breaches that compromised the data of well over 100 million victims. The earlier revelation about the financial costs of cyber attacks is damning in itself, but it’s crucial for brands — and consumers themselves — to recognize that data breaches can be unintentional or malicious, but in any case, they could affect millions of people.
Then, affected companies have to engage in damage control in an attempt to restore lost trust. Even when those entities put forth the effort, they may still find that customers behave differently following breaches.
More specifically, an April 2018 study examined the connection between consumer trust and spending. It involved respondents giving a trust score to businesses. The survey revealed that 15 percent of low-trust customers decreased how much they spent at companies. But, in cases of high instances of trust, the decrease in consumer spending was only 4 percent.
5. It Takes Months to Identify and Contain Breaches
If a person or business has a significant water leak in a well-used area, the problem is usually easy to spot and fix. But, it’s typically not so straightforward with cyber-related issues.
Research from 2018 published by IBM found that, on average, it takes 197 days to identify a breach and 69 days to contain it. Those timeframes give hackers plenty of time to do damage that may prove irreparable. Then, once headlines indicate how long a breach remained unnoticed, the reputational damage could be severely harmful, too.
Making cybersecurity a focal point this year could minimize the time spent looking for areas of concern within a network, especially if using artificial intelligence-based strategies that learn normal conditions and give warnings about deviations.
6. Cybercrime Is Extremely Profitable
Some criminals alter their methods once it becomes apparent that their current wrongdoings are no longer profitable. But, that probably won’t happen for a while concerning online-based crimes. Research from a criminology expert published in April 2018 highlighted how the worldwide revenues from cybercrime are at least 1.5 trillion annually.
The investigation talked about how cybercrime represents an interconnected web of profit possibilities with blurred lines between legal and illegal activities. If people don’t fight back against online criminals at both personal and organizational levels, hackers will have more opportunities than ever to continue making income while others suffer.
Failing to Focus on Cybercrime This Year Could Cause an Assortment of Issues
This list highlights some of the most prominent reasons why it’s essential to make cybersecurity a priority in 2019. Hackers get progressively more skilled at carrying out attacks, and they can cause significant catastrophes on unprotected or poorly defended
About the author
Kayla Matthews is a technology and cybersecurity writer, and the owner of ProductivityBytes.com. To learn more about Kayla and her re
(Security Affairs – 2019 Cybersecurity predictions, cyberattacks)
Some Americans like their pop musicians to be more accessible, less theatrical, and eccentric–and generally more desperate for the approval of their audience. Kate Bush, thankfully, has never seemed bothered by this need. She could leave the spotlight when she needed to, or leave the music business altogether for a time, and yet remain a creative force to be reckoned with for four decades now. Her legacy has permeated contemporary music since she appeared in 1978, then retired from the stage the following year after her first tour to focus solely on writing, recording, and making short musical films.
Her debut, The Kick Inside_, proved that an original new songwriter worth watching had arrived, and she delivered on the promise in ten studio albums and a career she seemed to sum up in the title of “This Woman’s Work,” from 1989’s _The Sensual World. It is work she has always done in her own delightfully odd, passionate, eccentrically British, theatrical, and deftly literary way, all qualities that have made her a massive star in the UK and a hero to artists like Tori Amos, Annie Lennox, Grimes, Florence and the Machine, and too many more to name.
Bush’s unusual traits also make her a perfect artist to pay tribute to in an orchestral setting, as Sweden’s Gothenburg Symphony has done in the 2018 concert also titled “This Woman’s Work” and featuring the very-Bush-worthy vocal talents of guest singers Jennie Abrahamson and Malin Dahlstrom. It’s “a towering tribute,” the Symphony writes, “with hit songs and pure poetry in special arrangements by Martin Schaub.” And it arrived to mark a special moment indeed: the 40th anniversary of the release of Bush’s brilliantly strange debut single “Wuthering Heights.” See the full performance at the top of the post and excerpted songs throughout, including Abrahamson’s cover of “This Woman’s Work,” above.
Appearing in the ghostly guise and ethereally high-pitched voice of Cathy Earnshaw, doomed heroine of Emily Bronte’s novel, Bush captivated millions in two videos that are now absolute classics. She drew on the mime theatrics of her teacher Lindsay Kemp, who previously mentored David Bowie, and gave us the indelible image of a woman possessed by weird imagination, uncanny musical talent, and some frightening dance moves. The images and sounds she created in just those 3 and a half minutes are iconic. Or, putting it a little differently in a short BBC documentary, John Lydon says, “Kate Bush and her grand piano& that’s like John Wayne and his saddle& her shrieks and warbles are beauty beyond belief.”
If you came to Bush later in her career, say during 1985’s huge_ Hounds of Love_, and somehow missed her unbelievable first fine art-rock performances on film, watch both the white and red dress versions first, then watch the Gothenburg Symphony’s glowing, career-spanning tribute to a woman who “laid the groundwork for [a] generation of performers,” as Marc Hirsh writes at NPR. Even though he is an American who does not care for Kate Bush, Hirsh can’t seem to help enumerating the very reasons she is so special to so many, and he features a number of her videos that demonstrate why she’s an artist her fans love “from the very core of their being.”
Kate Bush’s First Ever Television Appearance, Performing “Kite” & “Wuthering Heights” on German TV (1978)
The Largest Ever Tribute to Kate Bush’s “Wuthering Heights” Choreographed by a Flashmob in Berlin
Vincent van Gogh never went to Japan, but he did spend quite a bit of time in Arles, which he considered the Japan of France. What made him think of the place that way had to do entirely with aesthetics. The Netherlands-born painter had moved to Paris in 1886, but two years later he set off for the south of France in hopes of finding real-life equivalents of the “clearness of the atmosphere and the gay colour effects” of Japanese prints. These days, we’ve all seen at least a few examples of that kind of art and can imagine more or less exactly what he was talking about. But how did the man who painted Sunflowers and The Starry Night come to draw such inspiration from what must have felt like such exotic art of such distant a provenance?
“There was huge admiration for all things Japanese in the second half of the nineteenth century,” says the Van Gogh Museum’s visual essay on the painter’s relationship with Japan. “Very few artists in the Netherlands studied Japanese art. In Paris, by contrast, it was all the rage. So it was there that Vincent discovered the impact Oriental art was having on the West, when he decided to modernise his own art.”
Having got a deal on about 660 Japanese woodcuts in the winter of 1886-87, apparently with an intent to trade them, he ultimately held on to them, copied them, and even used their elements as backgrounds for his own portraits.
“My studio’s quite tolerable,” he wrote to his brother Theo, “mainly because I’ve pinned a set of Japanese prints on the walls that I find very diverting. You know, those little female figures in gardens or on the shore, horsemen, flowers, gnarled thorn branches.” More than a diversion, he saw in their radical difference from the rigorously realistic, convention-bound traditional European painting a way toward “the art of the future,” which he was convinced “had to be colourful and joyous, just like Japanese printmaking.” As he developed what he called a “Japanese eye” while living in Arles, “his compositions became flatter, more intense in colour, with clear lines and decorative patterns.”
The Van Gogh Museum has digitized and made available to download Van Gogh’s Japanese art collection, or at least most of them: you can read about the hundred or so “missing” works here, and you can view the 500 the museum has retained here. Every time you reload the front page, the selection it presents reshuffles; otherwise, you can browse the collection by subject, person and institution, technique, object type, and style. Some of the best-represented categories include landscape, actor print, spring, and female beauty. Whether the Japan-inspired Van Gogh (or colleagues who shared his interest, chiefly Paul Gauguin) succeeded in creating the art of the future is up to art historians to debate, but no one who sees his collection of Japanese art will ever be able to unsee its influence on his own work. Not that Van Gogh didn’t admit it himself: “All my work,” he wrote in a later letter to Theo, “is based to some extent on Japanese art.”
Nearly 1,000 Paintings & Drawings by Vincent van Gogh Now Digitized and Put Online: View/Download the Collection
The Van Gogh of Microsoft Excel: How a Japanese Retiree Makes Intricate Landscape Paintings with Spreadsheet Software
Enter a Digital Archive of 213,000+ Beautiful Japanese Woodblock Prints
Based in Seoul, Colin Marshall writes and broadcasts on cities, language, and culture. His projects include the book _The Stateless City: a Walk through 21st-Century Los Angeles _and the video series _The City in Cinema. Follow him on Twitter at @colinmarshall or on Facebook._
This is wonderful.
Conventional wisdom in IT security has long taught us that zero-day exploits are rare and that we need to be far more concerned with non-zero-days, which make up the vast majority of attacks.
This was true. In my experience, few if any security professionals still say this in this way. It makes the wrong statement.
This paradigm was challenged recently by Microsoft security researcher Matt Miller in an awesome presentation he did on the evolution of Microsoft Windows exploits and defenses for Microsoft’s last Blue Hat event on February 7.
Prior to seeing Miller’s presentation, I would have guessed that zero-days were still rare.
Don’t guess. Ask a professional.
The new data that Miller had collected declared that zero-days are actually the norm, and non-zero days are getting less common over time. He showed that in 2017, every actively exploited Microsoft vulnerability was first done using a zero-day attack. In 2012, that number was 52 percent and had been as low as 21 percent in 2008.
As it should. The threat landscape is wide. The number of platforms, packages, and programs in an environment continues to grow.
Needless to say, his findings have generated lots of discussion. If misunderstood, a reader might be forgiven for wondering how important a role patching plays if the vast majority of exploits have no patch. Here’s an excellent example of why you don’t want to take one data point to build a defense.
I have not seen any new discussion, at least among security veterans.
Most vulnerabilities are not exploited
Even though we are now learning about over 15,000 newly discovered public vulnerabilities a year, most are never actively exploited. According to Miller’s own data in the same presentation, just barely 0.02 percent (12 out of 588 Windows CVEs) were actively exploited. This data is backed up by other risk management companies, such as Kenna Security, which says that only 0.6 percent of all CVEs (not just Microsoft Windows CVEs) are ever exploited in the wild.
Like hundreds of other teenagers in late seventies England, Susan Ballion, better known as Siouxsie Sioux, embraced the anyone-can-do-it-ness of punk after seeing the Sex Pistols. In 1976, already a tastemaker in the scene, she threw a band together with Sid Vicious on drums, and with no practice, or even any songs, they got onstage, and improvised a 20-minute rendition of “The Lord’s Prayer.” There launched the career of a post-punk, dark pop legend, spanning that first anarchic gig, the infamous Bill Grundy TV appearance, some of the most influential British rock of the late-70s and 80s, and major tours and hits throughout the last three decades.
Despite the awards, star collaborations, and multi-generational influence, Siouxsie’s striking musical talent has often been given short shrift in the U.S. press. For example, a 1992 _Los Angeles Times _concert write-up after the release of her biggest U.S. hit, “Kiss Them for Me,” cast her as “the leader of a cult of weird chicks,” writes Liz Ohnanesian at Noisey, “in a review that spent five paragraphs on her looks and a whopping two on the music.” Maybe, “at that point, the band was used to that.” But it’s a serious oversight.
“Much has been written about the vocal range of artists like Freddie Mercury,” Dangerous Minds points out, “but not so much on the equally brilliant Siouxsie Sioux.” If the comparison seems stretched, consider another one: Kate Bush.
Though very different artists, both released debut albums in 1978 and became style icons who are as influential for their look as for their vocal prowess. Siouxsie, whose voice “developed from spiky, punky vocals to rich, powerful, and glorious textured tones& can hit the high notes and bring an unnerving warmth and menace to her lower range.”
With Siouxsie and the Banshees, The Creatures, and in her solo work, she has given cool, icy voice to gothic sentiments and images, conveying ache and fear and brutal beauty. In the videos here, listen to Siouxsie’s isolated vocals from 1988’s “The Killing Jar” (hear the original right above), an excellent example of “just how good she is.” Above, also hear her vocal track from “Hong Kong Garden,” her 1978 debut single, and “arguably the most important of the early post-punk hits,” writes Robert Webb at The Independent.
Listen to her sing 1985’s “Cities in Dust,” about the destruction of Pompeii, and below, hear “Kiss Them for Me,” a cryptic tribute to actress Jayne Mansfield and a song that made a new generation of Siouxsie and the Banshees fans when it came out in 1991. Siouxsie has attracted a newly devoted fanbase every decade since the 70s for her style, songwriting, and her voice, an instrument that deserves greater attention.
“These days,” she said in a 2007 interview, the legacy of punk has “almost been reduced to a fashion statement. I think there’s been a false sense of empowerment for women” in music. “Almost as if there’s that ever-present preoccupation with body form and image& not about expressing any style or intent.” Young artists looking for genuine inspiration will always find the real thing in Siouxsie’s impressive body of work.
via Dangerous Minds
Hear Siouxsie and the Banshee’s Raw & Completely Improvised First Show, with Sid Vicious on Drums (1976)
The Sex Pistols Make a Scandalous Appearance on the Bill Grundy Show & Introduce Punk Rock to the Startled Masses (1976)
I have a lengthy TODO.org of things I might eventually implement for Emacs, most of which are not exactly useful, are challenging to do or fulfill both conditions. A NES emulator fits all of these criteria neatly. I’ve kept hearing that they can run on poor hardware and learned that the graphics fit into a tiled model (meaning I wouldn’t have to draw each pixel separately, only each tile), so given good enough rendering speed it shouldn’t be an impossible task. Then the unexpected happened, someone else beat me to the punch with nes.el. It’s an impressive feat, but with one wrinkle, its overall speed is unacceptable: Mario runs with a slowdown of over 100x, rendering it essentially unplayable. For this reason I adjusted my goals a bit: Emulate a simpler game platform smoothly in Emacs at full speed.
Enter the CHIP-8. It’s not a console in that sense, but a video game VM designed in the 70ies with the following properties:
* CPU: 8-Bit, 16 general-purpose registers, 36 instructions, each two bytes large
* RAM: 4KB
* Stack: 16 return addresses
* Resolution: 64 x 32 black/white pixels
* Rendering: Sprites are drawn in XOR mode
* Sound: Monotone buzzer
* Input: Hexadecimal keypad
It’s perfect. Sound is the only real issue here as the native sound support in Emacs is blocking, but this can be worked around with sufficient effort. Once it’s implemented there’s a selection of almost a hundred games to play, with a few dozen more if you implement the Super CHIP-8 extensions. I’d not have to implement Space Invaders, Pacman or Breakout with gamegrid.el. What could possibly be hard about this? As it turns out, enough to keep me entertained for a few weeks. Here’s the repo.
First of all, I’ve located a reasonably complete looking ROM pack. It’s not included with the code as I’m not 100% sure on the legal status, some claim the games are old enough to be public domain, but since there are plenty of new ones, I decided to go for the safe route. Sorry about that.
Cowgod’s Chip-8 Technical Reference is the main document I relied upon. It’s clearly written and covers nearly everything I’d want to know about the architecture, with a few exceptions I’d have to find out on my own. Another helpful one is Mastering CHIP-8 to fill in some of the gaps.
To boot up a CHIP-8 game on real hardware you’d use a machine where the interpreter is loaded between the memory offsets #x000 and #x200, load the game starting at offset #x200, then start the interpreter. It would start with the program counter set to #x200, execute the instruction there, continue with the next instruction the program counter points to, etc. To make things more complicated there’s two timers in the system running at 60Hz, these decrement a special register if non-zero which is used to measure delays accurately and play a buzzing sound. However, there is no specification on how fast the CPU runs or how display updates are to be synchronized, so I had to come up with a strategy to accomodate for potentially varying clock speeds.
The standard solution to this is a game loop where you aim at each cycle to take a fixed time, for example by executing a loop iteration, then sleeping for enough time to arrive at the desired cycle duration. This kind of thing doesn’t work too well in Emacs, if you use sit-for you get user-interruptible sleep, if you use sleep-for you get uninterruptable sleep and don’t allow user input to be registered. The solution here is to invert the control flow by using a timer running at the frame rate, then being careful to not do too much work in the timer function. This way Emacs can handle user input while rendering as quickly as possible. The timer function would execute as many CPU cycles as needed, decrement the timer registers if necessary and finally, repaint the display.
Each component of the system is represented by a variable holding an appropriate data structure, most of which are vectors. RAM is a vector of bytes, the stack is a vector of addresses, the screen is a vector of bits, etc. I opted for using vectors over structs for simplicity’s sake. The registers are a special case because if they’re represented by a vector, I’d need to index into it using parts of the opcode. Therefore it would make sense to have constants representing each register, with their values being equal to the value used in the opcode. Initially I’ve defined the constants using copy-paste but later switched to a chip8-enum macro which defines them for me.
The built-in sprites for the hex digits were shamelessly stolen from Cowgod’s Chip-8 technical reference. They are copied on initialization to the memory region reserved for the interpreter, this allows the LD F, Vx instruction to just return the respective address. When implementing extended built-in sprites for the Super CHIP-8 instructions there was no convenient resource to steal them from again, instead I created upscaled versions of them with a terrible Ruby one-liner.
For debugging reasons I didn’t implement the game loop at first, instead I went for a loop where I keep executing CPU instructions indefinitely, manually abort with C-g, then display the display state with a debug function that renders it as text. This allowed me to fully concentrate on getting basic emulation right before fighting with efficiency concerns and rendering speed.
For each CPU cycle the CPU looks up the current value of the program counter, looks up the two-byte instruction in the RAM at that offset, then executes it, changing the program counter and possibly more in the process. One unspecified thing here is what one does if the program counter points to an invalid address and what actual ROMs do in practice when they’re done. Experimentation showed that instead of specifying an invalid address they fall into an infinite loop that always jumps to the same address.
Due to the design choice of constantly two-byte sized instructions, the type and operands of each instruction is encoded inline and needs to be extracted by using basic bit fiddling. Emacs Lisp offers logand and ash for this, corresponding to &, << and >> in C. First the bits to be extracted are masked by using logand with an argument where all bits to be kept are set to ones, then the result is shifted all the way to the right with ash using a negative argument. Take for example the JP nnn instruction which is encoded as #x1nnn, for this you’d extract the type by masking the opcode with #xF000, then shift it with ash by -12. Likewise, the argument can be extracted by masking it with #x0FFF, with no shift needed as the bits are already at the right side.
A common set of patterns comes up when dissecting the opcodes, therefore the chip8-exec function saves all interesting parts of the opcode in local variables using the abbreviations as seen in Cowgod’s Chip-8 technical reference, then a big cond is used to tell which type of opcode it is and each branch modifies the state of the virtual machine as needed.
Nearly all instructions end up incrementing the program counter by one instruction. I’ve borrowed a trick from other emulators here, before executing chip8-exec the program counter is unconditionally incremented by the opcode size. In case an instruction needs to do something different like changing it to an jump location, it can still override its value manually.
To test my current progress I picked the simplest (read: smallest) ROM doing something interesting: Maze by David Winter. My debug function printed the screen by writing spaces or hashes to a buffer, separated by a newline for each screen line. After I got this one working, I repeated the process with several other ROMs that weren’t requiring any user input and displayed a (mostly) static screen. The most useful from the collection was “BC Test” by BestCoder as it covered nearly all opcodes and tested them in a systematic fashion. Here’s a list of other ROMs I found useful for testing other features, in case you, the reader, shall embark on a similar adventure:
* Jumping X and O: Tests delay timer, collision detection, out of bounds drawing
* Sierpinski triangle: Slow, tests emulation speed
* Zero: Animation, tests rendering speed (look for the flicker)
* SC Test: Tests nearly all opcodes and a few Super CHIP-8 ones
* Font Test: Tests drawing of small and big built-in sprites
* Robot: Tests drawing of extended sprites
* Scroll Test: Tests scrolling to the left and right
* Car Race Demo: Tests scrolling down
* Car: Tests emulation speed in extended mode
* Emutest: Tests half-pixel scroll, extended sprites in low-res
Debugging and Analysis
Surprisingly enough, errors and mistakes keep happening. Stepping through execution of each command with edebug gets tiring after a while, even when using breakpoints to skip to the interesting parts. I therefore implemented something I’ve seen in Circe, my preferred IRC client, a logging function which only logs if logging is enabled and writes the logging output to a dedicated buffer. For now it just logs the current value of the program counter and the decoded instruction about to be executed. I’ve added the same kind of logging to a different CHIP-8 emulator, chick-8 by Evan Hanson from the CHICKEN Scheme community. Comparing both of their logs allowed me to quickly spot where they start to diverge, giving me a hint what instruction is faulty.
Looking through the ROM as it is executed isn’t terribly enlightening, it feels like watching through a peephole, not giving you the full picture of what’s about to happen. I started writing a simple disassembler which decodes every two bytes and writes their offset and meaning to a buffer, but stopped working on it after realizing that I have a much more powerful tool at hand to do disassembly and analysis properly: radare2. As it didn’t recognize the format correctly, I only used its most basic featureset for analysis, the hex editor. By displaying the bytes at a width of two per row and searching for hex byte sequences with regex support I was able to find ROMs using specific opcodes easily.
Later after I’ve finished most of the emulator, I started developing a CHIP-8 disassembly and analysis plugin using its Python scripting support. I ran into a few inconsistencies with the documentation, but eventually figured everything out and got pretty disassembly with arrows visualizing the control flow for jumps and calls.
Later I discovered that radare2 actually does have CHIP-8 support in core, you need to enable it explicitly by adding -a chip8 to the command line arguments as it cannot be auto-detected that a file is a CHIP-8 ROM. The disassembly support is decent, but the analysis part had a few omissions and mistakes leading to less nice graphs. By using my Python version as basis I’ve managed improving the C version of the analysis plugin to the same level and even surpassed it as the C API allows adding extra meta-data to individual instructions, such as inline commentary. There is a pending PR for this functionality now, I expect it to be merged soon.
For maximum speed I set up firestarter to recompile the file on each save, added the directory of the project to load-path, then always launched a new Emacs instance from where I loaded up the package and emulated a ROM file. This is ideal if there isn’t much to test, but it’s hard to detect regressions this way. At some point I decided to give the great buttercup library another try and wrote a set of tests exercising every supported instruction with all edge cases I could think of. For each executed test the VM is initialized, some opcodes are loaded up and chip8-cycle is called as often as needed, while testing the state of the registers and other affected parts of the machinery. It was quite a bit of grunt work due to the repetitive nature of the code, but gave me greater confidence in just messing around with the code as retesting everything took less than a second.
Make no mistake here though, excessively testing the complicated parts of a package (I don’t believe it’s worth it testing the simple parts) is in no way a replacement for normal usage of it which can uncover completely different bugs. This is more of a safety net, to make sure code changes don’t break the most basic features.
Retrospectively, this was quite the ride. Normally you’d pick a suitable game or multimedia library and be done, but this is Emacs, no such luxuries here. Where we go we don’t need libraries.
My favorite way of drawing graphics in Emacs is by creating SVG on the fly using the esxml library. This turned out to be prohibitively expensive, not only did it fail meeting the performance goals, it also generated an excessive amount of garbage as trees were recursively walked and thrown away over and over again. A variation of this is having a template string resembling the target SVG, then replacing parts of it and generating an image from them. I attempted doing this, but quickly gave up as it was too bothersome coming up with suitable identifiers and replacing all of them correctly.
I still didn’t want to just drop the SVG idea. Considering this was basically tiled graphics (with each tile being an oversized pixel), I considered creating two SVG images for white and black tiles respectively, then inserting them as if they were characters on each line. The downside of this approach was Emacs’ handling of line height, I couldn’t figure out how to completely suppress it to not have any kind of gaps in the rendering. gamegrid.el somehow solves it, but has rather convoluted code.
At this point I was ready to go back to plain text. I remembered that faces are a thing and used them to paint the background of the text black and white. No more annoying gaps. With this I could finally work and started figuring out how to improve the rendering. While the simple solution of always erasing the buffer contents and reinserting them again did work, there were plenty of optimization possibilities. The most obvious one was using dirty frame tracking to tell if the screen even needed to be redrawn. In other words, the code could set a chip8-fb-dirty-p flag and if the main loop discovered it’s set, it would do a redraw and unset it. Next up was only redrawing the changed parts. For this I’d keep a copy of the current and previous state of the screen around, compare them, repaint the changed bits and transfer the current to the previous state. To change the pixels in the buffer I’d erase them, then insert the correct ones.
The final optimization occurred me much later when implementing the Super CHIP-8 instructions. It was no longer possible to play games smoothly at quadrupled resolution, so I profiled and discovered that erasing text was the bottleneck. I considered the situation hopeless, fiddled around with XBM graphics backed by a bit-vector and had not much luck with getting them to work nearly as well at low resolution. It only occurred me by then that I didn’t try to just change the text properties of existing text instead of replacing text. That fixed all remaining performance issues. Another thing I realized is that anything higher-resolution than this will require extra trickery, maybe even involving C modules.
Garbage Collection Woes
Your code may be fast, your rendering impeccable, but what if every now and then your bouncing letters animation stutters? Congratulations, you’ve run into garbage collection ruining your day. In a language like C it’s much more obvious if you’re about to allocate memory from the heap, in a dynamic language it’s much harder to pin down what’s safe and what’s not. Patterns such as creating new objects on the fly are strictly forbidden, so I tried fairly hard to avoid them, but didn’t completely succeed. After staring hard at the code for a while I found that my code transferring the current to the old screen state was using copy-tree which kept allocating vectors all the time. To avoid this I wrote a memcpy-style function that copied values from one array to another one.
Another sneaky example was the initialization of the emulator state which assigned zero-filled vectors to the variables. I noticed this one only due to the test runner printing running times of tests. Most took a fraction of a millisecond, but every six or so the test took over 10 milliseconds for no obvious reason. This turned out to be garbage collection again. I rediscovered the fillarray function which behaves much like memset in C, used it in initialization (with the vectors assigned at declaration time instead) and the pauses were gone. No guarantees that this was the last of it, but I haven’t been able to observe other pauses.
If your Emacs has been compiled with sound support there will be a play-sound function. Unfortunately it has a big flaw, as long as the sound is playing Emacs will block, so using it is a non-starter. I’ve initially tried using the visual bell (which inverts parts of the screen) as a replacement, then discovered that it does the equivalent of sit-for and calling it repeatedly in a row will in the worst case of no pending user input wait as long as the intervals combined. There was therefore no easy built-in solution to this. To allow users to plug in their own solution I defined two customizable functions defaulting to displaying and clearing a message: chip8-beep-start-function and chip8-beep-stop-function.
The idea here is that given a suitable, asynchronous function you could kick off a beep, then later stop it. Spawning processes is the one thing you can easily do asynchronously, so if you had a way to control a subprocess to start and stop playing a sound file, that would be a good enough solution. I then remembered that mplayer has a slave mode and that mpv improved it in a multitude of ways, so I looked into the easiest way of remote controlling it. It turns out that mpv did away with slave mode in favor of controlling it via FIFO or a socket. To my surprise I actually made it work via FIFO, the full proof of concept can be found in the README.
The CHIP-8 supports two ways of checking user input: Checking whether a key is (not) pressed (non-blocking) and waiting for any key to be pressed (blocking). Doing this in a game library wouldn’t be worth writing about, but this is Emacs after all, there is only a distinction between key up and down for mouse events. After pondering about this issue for a while I decided to fake it by keeping track of when keys have been last pressed in a generic key handler function, then comparing that timestamp against the current time: If it’s below a reasonable timeout, the key is considered pressed, otherwise it isn’t.
Solving the other problem required far more effort. The emulator was at this point sort of a state machine as I’ve tracked whether it was running with a boolean variable to implement a pause command. I’ve reworked the variable and all code using it to be mindful of the current state: Playing, paused or waiting for user input. This way the command merely changed the current state to waiting for input, set a global variable to the register to be filled with the pressed key and set the stage for the generic key handler function to continue execution. If that function detected the waiting state and a valid key has been pressed, it would record it in the respective register and put the emulator into playing state again.
Actually testing this with a keypad demo ROM unveiled a minor bug in the interaction between the main loop and the redrawing logic. Remember that a number of CPU cycles were executed, then a redraw was triggered if needed? Well, imagine that in the middle of the CPU cycles to be executed the state were changed to waiting and the redraw never happened! This would produce an inconsistent screen state, so I changed it to do a repaint immediately. Furthermore, if the state changed to waiting, the loop would still execute more cycles than needed (despite it being a blocking wait), therefore I had to add an extra check in the main loop’s constant amount of cycling whether the state changed and if yes, skeep the loop iteration alltogether.
At this point I was pretty much done with implementing the full CHIP-8 feature set and started playing games like Tetris, Brix and Alien.
Yet I wasn’t satisfied for some strange reason. I probably longed for more distraction and set out to implement the remaining Super CHIP-8 instructions. Unlike the main instruction set these weren’t nearly as well documented. My main resource was a schip.txt file which briefly describes the extra instructions. The most problematic extension is the extended mode which doubles the screen dimensions, requiring a clever way to draw a bigger or smaller screen whenever toggled. There are two ways of implementing such a thing: Drawing to one of two separate screen objects and painting the correct one or alternatively, always drawing to a big screen and rendering in a downscaled mode if needed. For simplicity’s sake I went with the first option.
The extra scroll extensions allow game programmers to efficiently change the viewport (though for some reason they forgot about an instruction scrolling up). My challenge here was to change the screen’s contents in-place, for this to be done correctly extra care was necessary to not accidentally overwrite contents you needed to move elsewhere. The trick here is to iterate in reverse order over the screen lines if necessary.
A few more instructions and optimizations later and I was ready to play the probably silliest arcade game ever conceived, Joust. The sprites in the picture below are supposed to be knights on flying ostrichs trying to push each other down with their lances, but they look more like flying rabbits to me.
Writing an emulator gives you great insight in how a machine actually works. Details like memory mapping you glossed over feels far more intuitive once you have to implement it yourself. One of the downsides is that I didn’t play games for my own enjoyment, but to further improve the emulator and understand the machine.
A few games and demo ROMs revealed bugs in the emulator, such as how to deal with drawing sprites that go outside the boundaries. Cowgod’s Chip-8 Technical Reference tells you to do wrap-around, but Blitz by David Winter seems to think otherwise, when rendered with wrap-around the player sprite collides immediately into a pixel on the edge and the “GAME OVER” screen is displayed. I decided in this case to forego that recommendation and clip the rendering to the screen edges.
It’s not always easy to make such decisions. Some quirks seem fairly reasonable, such as preferrably setting the VF flag to indicate an overflow/underflow condition for arithmetic, although it’s not always specified. Some quirks seem fairly obscure, such as the interpretation of Super CHIP-8 extensions in low-resolution mode: A demo insists that instead of drawing a high-resolution 16 x 16 sprite it should be drawn as 8 x 16 instead. As this doesn’t appear to affect any game and requires significant support code I decided against implementing it. In one case I was conflicted enough between the different interpretation of bit shifting operators that I introduced a customizable to toggle between both, with the incorrect, but popular behavior being the default.