Synopsis: We discuss the process of starting, learning from and balancing a new software (open source) project.
There is an inherent difficulty in starting and maintaining some ambitious software projects; wearing infinite hats. Assuming you're starting from a state of relatively few resources, you must become: strategic planner, developer, recruiter, designer, architect, evangelist, researcher, etc. The list goes on. Hopefully, you're in the situation where you can openly and readily enlist the help of others to contribute, but more than likely, you have to first get your project to the point where others can understand, believe in, and work on a project. This is probably a good opportunity anyway, as it gives you the chance to more intimately understand the other domains involved in ideating, designing and then shipping something.
Put on your dreamer's cap
It starts, as it should, with vision. The fundamental building block of everything else is the belief in something. You should believe that what you're working on should exist for some reason; whatever that reason is or however it manifests itself is ultimately up to you, but you should believe in it. There will be difficult and trying times which will test your capability to continue with or finish it, and if you can anticipate your project taking some serious investment - make sure you're mentally prepared for that investment.
Write down your vision, whether it be a radical new system, a simple library or framework, a complex service - but make sure that it conveys your intention. The reason you're working on your new widget is driven by something, some need, and you should intimately understand this drive. Your vision will help to be your North star.
Throw on your best toque blanche
You have your vision. A good next step is to brainstorm how you could effectively implement your vision. This is important to do, because depending on how your brain functions, you either have a ton of preconceived notions/assumptions that can influence the rest of your creation process, or you have no idea where to proceed from vision. Or maybe somewhere in between - who knows.
The point is to get down your ideas and let go of them; whether they regard your value propositions, who your user (or customer) segments are, how it should be engineered, etc. Whenever we get started on something, we have to operate on some basis or system of assumptions in order to get anywhere meaningful, but assumptions are dangerous. They prevent us from seeing details, issues, or opportunities we would otherwise have seen if we followed a scientific(ish) process of some kind.
Now that you've let go of your assumptions: it is perhaps time to adopt a framework for research, implementation, etc. There are numerous methodologies which you can use or learn to begun turning your vision into coherent hypothesis. Alex Osterwalder's business model canvas and/or value proposition canvas can help you to formulate where your project fits into the grand scheme of things for your users or customers (as well as how it might function as a business). I'm a proponent of treating your project like a business.
For myself, I've realized early working on Unifier (forgive the site, I just put it up yesterday) that what I'm really after is both a combination of improvement in the user experience for my end users, as well as an increase in functional capabilities (innovation). In my particular case, my Minimum Viable Product must have a user experience superior to that of status quo options.
In this situation, I've found it necessary to also adopt a design process and framework for understanding and constructing the ideal experience to help my end users accomplish their goals. Enter Goal-Directed Design or any number of design / user experience frameworks ... though I've really been enjoying "About Face: The Essentials of Interaction Design, 4th Edition" (Robert Reimann; Christopher Noessel; Alan Cooper; David Cronin).
The point I'm trying to convey is that a system(s) or framework can help guide you along the process of creating something new. You will find at times that it can be rather difficult to know how or where to proceed, and having a reasonable system to follow starting points can prevent paralysis analysis.
Adorn your handy, helpful [life-saving] hard-hat
You can prototype or test any number of different ways: using code, wireframes, paper and pencil, Photoshop, etc. It is important that you actually do it. You need to show both yourself and others a translation of your vision and research; prove to yourself that it can exist. Construct a prototype; just don't get too married to it. The idea is just that you have something to show - it doesn't need to be the final product (or even have functionality).
More to come ...