Feedback, Contributions, and Project Plans
Hi all, Lev here.
I'm writing with some feedback about DuckDuckHack. Specifically, my feedback is concerned with the task of increasing contribution through DuckDuckHack, whether by creating, suggesting, or discussing Instant Answers.
Before I get into my critique, I'd like to talk about what DuckDuckHack gets right. From what I’ve seen, the aspects of the DuckDuckHack ecosystem that are the strongest are the development environment, the Slack community, and the aesthetic design.
I recently submitted two small Pull Requests (one to fix a Goodie, and one to create a Goodie), and I was very impressed with the Codio setup - it was incredibly hassle-free to start playing around in a working environment. With Duckpan, running tests, querying Instant Answers, and spinning up the local server were all seamless experiences. In my experience, this is not the case for many open source projects, even large and popular projects.
I mention the Slack community as something DuckDuckHack gets right because I've found it fairly helpful and responsive, especially when compared to the forum. Rarely does a question go unanswered on Slack, while on the DuckDuckHack forum, a brief glance at the landing page shows a number of well written posts with no responses.
The last aspect of DuckDuckHack that I think is done really well is the design. By this I mean the actual aesthetic design, not the UX (which I have some feedback about below). I really like the color scheme, the featured Instant Answer is a neat idea, and the custom icons are visually appealing. Like duck.co and duckduckgo.com, the design of DuckDuckHack is crisp and aesthetically pleasing.
Constructive Feedback Overview
DuckDuckHack has a great code contribution environment, a beautiful website, and an engaged community on Slack. When phrased like that, it sounds as if there should be no problem attracting contributors to DuckDuckHack. A new user lands on the beautiful site, talks to the community on Slack, and finally submits some changes made through the seamless development environment.
The devil is in the details: although the site is aesthetically pleasing, the main calls to action are ineffective. And despite the engaged Slack community, there are issues with the Discourse forum - it is promoted more heavily on DuckDuckHack, despite lacking almost any community interaction. And the contribution process, although positive in the code environment, is rough around the edges, especially when it comes to finding appropriate Instant Answers to build or fix.
I'll start with the UX of the DuckDuckHack landing page. When you first visit the site, you see a call to action that says "Get Started." When you click it, you are taken down past the fold to a short blurb about DuckDuckHack and another call to action that leads you to the DuckDuckHack Tasks page. I think a Tasks page makes sense to have, but in its current state it is only minimally useful.
It only displays a few GitHub issues, and it is unclear why these are listed instead of all of the issues tagged as "Priority: High" (or "Category: Highest Impact Tasks," which to me seems to be a confusingly similar label). I believe this Tasks page should either show all GitHub issues, all "Priority: High" issues, or maybe even just link directly to the GitHub issues so that users can search and sort for themselves.
But most importantly, this should not be the primary call to action on the page. It is not appropriate for first time contributors because they may not be able to code and will not understand it, or they may be programmers who should be directed to the documentation. It is also not very useful for experienced contributors - they likely know where to go and are not navigating through DuckDuckHack’s landing page.
Beneath the primary call to action are two secondary calls to action, which could also use some improvement.
If you click "Current Projects and Instant Answers," you are taken to a tag on the forum called "overview-post" - not only is this tag not representative of current projects (the last post is from a month ago and has 0 replies), but it doesn't really have anything to do with Instant Answers. A button that says "Current Projects" could link to the Tasks page, GitHub Issues or GitHub Pull Requests - that phrase isn't used anywhere else so it's not obvious what it refers to. And a button that says "Instant Answers" should definitely link to duck.co/ia, the Instant Answers main page. It's unclear why a button that has both of those phrases in it links to neither of the pages semantically related to either phrase.
If instead, you click "Become a Leader", the other secondary call to action, you would find that it links to a locked forum post that says the "Programming Mission" project has come to an end. This definitely should not still be linked to on the main DuckDuckHack page.
The last call to action is a button that lets you sign up for the DuckDuckHack Slack group, which is perfectly fine.
A quick look at the user’s journey show that practically any path they take will confuse them - "Current Projects and Instant Answers" doesn't seem to be about Current Projects or Instant Answers, "Become A Leader" links to a locked post that does not seem to be relevant anymore, and "View Open Tasks" takes you to a page that is supposed to show important GitHub Issues but seems to be infrequently updated and confusingly similar to the "Priority: High" and "Category: Highest Impact Tasks" labels on GitHub.
To recap my first piece of constructive feedback, the landing page for DuckDuckHack has a total of four calls to action, three of which are either easy to misinterpret (“Current Projects and Instant Answers”), irrelevant (“Become a Leader”), or unmaintained (“View Open Tasks”). While this sounds like substantial criticism, I believe that the fix for this issue is actually pretty simple.
I've created a quick demo of what I think the calls to action should be on the landing page. In this version, the three calls to action are "Suggest an Instant Answer," which links to the Instant Answer Creation Wizard and the list of Instant Answers, "Discuss Your Ideas" which links to the Forum and the Slack communities, and "Build an Instant Answer," which links to the documentation and the GitHub Issues page (or Tasks page if that is fixed).
This way, all major categories of users have easy access to the links that actually matter to them. New users are able to suggest or browse ideas, whereas formerly they'd have to look in the footer for both of those links. They can also easily join Slack or the Discourse forum. And lastly, contributors (both new and old) have quick links to the documentation or the list of issues that they could tackle. It's a win-win-win!
Another small improvement I made was visually balancing the calls to action. Before, there was a hierarchy of calls to action resulting from their triangular layout. As I mentioned previously, the top call to action (“View Open Tasks”) should not be the primary call to action - it links to a page that seems mostly unmaintained and even if it were maintained would not be appropriate for first time contributors. To fix this, I adopted a more balanced layout for the three calls to action. I also ensured that the three of them had a consistent number of links and a similar amount of descriptive text so that they have better vertical alignment. Lastly, I fixed the responsiveness bug that caused the calls to action to overlap on mobile screens.
As I have already forked the DuckDuckHack repository and made a small demo of my proposed changes to the calls to action, the next step is to collaborate with the other DuckDuckGo employees involved in this site. I will discuss the visual changes with the Design team and the copy with the Marketing team so that we can improve the current landing page as a team.
I estimate this should take less than a week. This timeframe is part of the reason I think this contribution would be so valuable - it follows the 80/20 rule, which is to say that it fixes a lot of the User Experience problems on the DuckDuckHack landing page with minimal effort. Instead of rewriting the site or coming up with an entirely new design, I think in this scenario (and most other scenarios as well) it is better to iteratively improve the existing product. I believe this is especially true for DuckDuckHack, as it largely succeeds in its purpose and is only hampered by a few small issues.
Before any code is contributed, a developer usually will find an appropriate bug or feature request to implement. When I first looked at how to contribute, I was confused by the Tasks page on DuckDuckHack, which seemed inactive and full of issues that were beyond my understanding or were already being tackled by someone else. I think that a good solution would be to tweak the existing infrastructure such that the Tasks page displays all GitHub Issues that have a specific label.
As evidence for my position that the Tasks page is ineffective in its current state, I’ve gone through the Slack group and counted three people in the last week who have asked for help because they were unsure how to contribute. All three of them quickly got responses from community members, so their specific experiences are not the issue. The issue is that presumably for every user who is confused about how to contribute and joins the Slack asking how they can help, there are others who either didn’t see the Slack group or simply gave up once they couldn’t find information about contributing.
This feedback I have relates back to my earlier criticism of the calls to action - I think that having a call to action for developers with a link to the documentation and the Tasks would help mitigate this, but to really address the problem, the Tasks page will have to be made more effective.
The Tasks page is an excellent idea that seems to suffer from a lack of maintenance.
I think that having a central collection of all the important tasks makes a lot of sense; contributors would easily be able to see what needs to be prioritized and how they could effectively contribute. However, what separates the GitHub issues on this page from the ones labelled “Priority: High” or "Category: Highest Impact Tasks" is unclear to me, and presumably to others.
A simple way to improve this page would be to use the GitHub API to populate a table of issues. This table could be limited to high priority issues with a specific label, or it could aggregate all issues from the key DuckDuckHack repositories and sort or filter them using other criteria. This second option makes more sense because it would allow contributors to filter all issues by the “Low-Hanging Fruit” or “Difficulty: Low” labels.
From a quick look at the DuckDuckHack repository, it seems to be currently using the GitHub API to generate the table of issues. The main problem is that the internal DuckDuckGo service (which itself is getting issues from the GitHub API) is only returning a few issues - although they all share the "Category: Highest Impact Tasks" label, I’m not sure why these specific issues are being returned and not all issues with that label.
I think the simplest approach would be to have the Tasks page automatically use the GitHub API to display issues that are marked with a specified label - this could be an existing label like “Priority: High” or a new one, like “DDH Task”. This would require access to the DuckDuckHack repository (either through write access or through a staff member approving my changes) and access to the backend service that returns issues with the GitHub API (located at https://duckduckhack.com/open_issues/).
Because this infrastructure already exists, it shouldn’t take long to tweak it so that it looks for all issues with a certain label. The other main improvement would be to make the list sortable and to talk to community leaders about how to label an issue such that it would appear on the Tasks page. I estimate that, in total, this would take a week.
Next up is the forum. I think the forum has a valuable purpose - Slack is not always the best place for discussion, especially for longer posts and conversations that span many weeks or months. However, the current forum call to action has a confusing label and takes you to the "overview-post" tag. Instead, it should be labelled as "Join our Forum" and take you to the home page of the forum. Also, the link to the forum in the footer takes the user to a Getting Started post. This post is actually very informative, but the forum link should still probably link to just the forum homepage, with a stickied Getting Started post at the top. As an aside, this Getting Started is followed by a Staff comment that offers a link for people who would like to contribute but are not programmers. This link goes to the “how-to-contribute” tag, which is mostly filled with forum posts that are only relevant to programmers (primarily language overviews).
As for the actual forum itself, the main issue that I see is engagement. This is in reference to user engagement, but perhaps more importantly, staff engagement. For example, one post on the first page asks about reporting incorrect translation tokens on duck.co. It has no responses, despite being a seemingly important question for DuckDuckGo staff. Perhaps no one on the staff saw it, or perhaps someone saw it and didn't think it was an issue, but the forum feels lifeless when a serious question goes unanswered or unseen for more than two weeks.
As of now, four of the twelve most recent posts on the forum are from prospective DuckDuckGo employees, another is someone who is confused as to the forum's purpose and is directed towards the DuckDuckGo subreddit, and another five posts have zero replies. The forum is meant to be a place for longer form conversation about contributing to DuckDuckGo, but it seems like many users are confused by its purpose and others get no responses. I think that you could make the argument to move the function of the forum to Slack, but I think it might also be worth considering Zulip. Zulip is similar to Slack in its live chat functionality, but has full featured conversations (unlike the basic thread support in Slack) and is also open source (which seems fitting as a place for open source discussion).
That being said, the particular forum or chat application used is not the crux of the issue. I think even the existing forum would be a lot more useful if external descriptions of the forum were more accurate and there were more interaction from DuckDuckGo staff.
My proposed contribution is an investigation of the current communication tools used by contributors to DuckDuckHack. In examining these mediums, I noticed that there is a significant point of weakness in the current ecosystem. The Slack chat is relatively responsive, and contributors seem able to communicate effectively through GitHub Issues. However, the Discourse forum does not appear to be fulfilling its purpose. Almost all of the recent forum posts are either from prospective employees, are told they are posting in the wrong place, or have no responses.
In my feedback, I speculated that the forum’s problems have less to do with its purpose and more to do with its execution. The forum is a good idea, but it currently lacks the community/staff engagement and appropriate marketing to be successful. On the DuckDuckHack landing page, the only link to the forum is hidden away in the footer. I amended this deprioritization in my landing page redesign, but even with more exposure, the forum needs more active engagement. Luckily, there are a few options for approaching this issue, which I have outlined below in my project plan.
The initial phase of this project will involve discussing the forum with stakeholders (members of the Slack group, members of the Discourse forum, and GitHub contributors). I posit that these discussions will provide data points when deciding what to do about the forum.
On that note, I think there are three main options to choose from. The first would be to better communicate to users the purpose of the forum (so as to reduce posts that are better suited to the subreddit) and to increase DuckDuckGo staff involvement. This change would likely require copy changes in order to explain the purpose of the forum and some sort of internal discussion among DuckDuckGo employees to encourage them to at least occasionally check the forum for unanswered questions, ideas, etc. This option would likely take 1-2 weeks, and is the least involved option.
It is important to note, however, that the timeframe for this (and all the following options) is ongoing. Although there is some initial development time, there is also a longer unbounded component, as any option chosen needs continuous input from staff in order to create an active communication environment.
The second option would be to get rid of the forum entirely. I think this option, though slightly more radical, is a fair option to consider. Getting rid of the forum would require some changes to DuckDuckHack and duck.co in order to replace old links to the forum, but besides that would mostly require organizational changes. The Slack channel would need to be clearly explained as a place for open source discussion, and contributors would be pointed there instead of the forum. This would only take a week at most to do, but would require discussion in the community about how to use the Slack group more effectively.
The last option would be to remove the forum and replace it with something else other than the existing Slack group. The benefit of this approach is that it would be possible to pick a replacement that has the benefits of long form conversation (like the forum) with the benefits of real time chat (like Slack). I propose Zulip as an alternative - it has live chat but also full-featured threads that are easy to refer back to (unlike Slack’s recent addition of threads as a mostly a second-class feature).
The main draw of Zulip is that it could potentially replace both the Slack group and the Discourse forum. It could have static conversations for “Getting Started” and other important posts like the forum does, while still maintaining the real-time chat aspect of Slack. In addition, it is open source, which seems especially apt as a place for contributors to discuss their open source work on DuckDuckHack. This alternative would likely take the longest and be the most involved, though again, the main challenge would not be setting up Zulip or replacing links, but acclimating the community to the change.
Based on the quantity of constructive feedback that I’ve given, you’d think DuckDuckHack was in a lot of trouble. But really, all of my critiques are centered around one idea, which is to polish what already works. In my view, the strongest aspects of DuckDuckHack are its design, its contribution process, and its open source communication. The user experience of the site, which in some respects is a function of the design, just needs a couple of small changes to the calls to action for considerable improvement. The contribution process is incredible once developers can get up and running; they just need a gentle nudge towards the documentation and a way to easily find important tasks. And the open source contributors have a remarkably engaged Slack community and are quite active on GitHub - it seems that only the forum is struggling in this domain. These small changes could make a critical difference because they connect pieces of the DuckDuckGo ecosystem that already excel.
Lastly, I just want to briefly share my conviction that no change is made in a vacuum. For every proposal I’ve made, there is a need to talk to the people who are affected by it. The designers who drafted the latest version of DuckDuckHack should be consulted about my ideas for the calls to action, and I should discuss my proposed changes with the marketers who wrote the original copy. The developers who wrote the integration into GitHub’s API for the Tasks page and designed the Task page itself need to be involved in the process of improving it. Likewise, the community members, contributors, and staff who communicate through the Discourse forum and the Slack group should all be involved in the discussion around revitalizing the forum.
As a brief closing thought, I’d like to pose a couple of questions that I considered while writing this. First, I’m curious what criteria the GitHub Issues service uses to fetch GitHub Issues to fill the Tasks page. It’s unclear to me why the particular issues on that page have been selected over other high priority issues. Secondly, I’m wondering if there’s ever been a community survey asking for feedback about DuckDuckHack. I hope the feedback from this job application process will provide some valuable insight, but I imagine there would be a lot to learn from asking other contributors why they keep contributing - or why they’ve stopped.
Thank you for your consideration; I look forward to discussing these ideas further.