Conferences are difficult for academics. We need them for publications and for networking, but nobody really likes them. The fun part is always the stuff that goes on at the same time, the conversations, the parties, the walks, the runs, the more-or-less planned meals, the trips back and forth - that's what feels like it makes it all worth the trip. We still go, because no matter how much we may dislike it, the next time we need a reference we remember that we heard something on that conference there... and then we have a way to track the argument down, and avoid repeating other people's work, or to go a step further ourselves.
However, since we all need them, but don't really love them, everybody have an opinion about them. So did I, many years ago, when I first started attending. Then I started organising them, and now I just thank every conference organiser as long as they are not obviously scamming. So for all of you who are unhappy about your current conferences, aside from perhaps looking around to check if there's something fun going on elsewhere, these are my recommendations.
Start organising seminars with colleagues. Just small stuff. One day, everybody cordially invited, a relevant topic for the group, but no call or reviews.
This lets you practice how to book rooms, how to make a schedule, find funding or a space for lunch, if everybody pays for themselves, how to keep time, and how to keep track of who are there and who are not able to come.
Next step is inviting a speaker. Same learning outcomes as above, but with the added complexity of inviting somebody from outside, somebody as many of your mates find interesting as possible. Done that? Good, next step.
Now you send out a call for your one day seminar, which means you need to practice writing a CFP (call for papers), find reviewers, create a double blind review process, follow up the reviews, send responses to the people who sent their papers in, including both acceptances and rejections. Once people arrive this time, they may not be from your little circle of friends and colleagues, so they need directions, a website with the program on and their paper on the program to prove for their institution that they need travel funding, preferably also a proceedings for use in their further struggle to be hired in academia, they have special dietary needs, they have disabilities, they need help to find a place to live - in general, you have suddenly upped the game seriously. But you are still within a one-day, one-track conferences. It's fine, it's still real, but it's still manageable out of your own computer on your own time.
But then this catches on, and you think "huh, we had to reject so many great papers, why don't we increase this to a two day conference with multiple tracks." Now you start having so many submissions that you no longer manage to keep track of every author and every reviewer, and you need to find a good system for managing the papers. There are several out there, and they all do the same thing: help you set up a place to submit the papers, register presenters, register reviewers, send out reviews, collect them, compare, make decisions and send out the decisions. Then you need to get the papers de-anonymised, the abstracts updated, and you need to put papers into sessions and find chairs for the sessions, create an even more elaborate program, deal with all the people who needs changes, handle the complaints about the reviews, etc etc. And you need to create a set of proceedings, that needs to be published, probably at this stage by your institution, and you need a stable website for it, because you are now doing something that impacts all those other people.
And then you start getting feedback. This is when either swear never to do this again, or you just swallow all the acid the feedback generates, and decide to learn from it and go on. Because at this point your work had made a jump from being fun and interesting for a small group, to being important enough that others care about what you do. And that means they also care about how you do it, whether you invite the right speakers, serve the right kind of food, think about the environment, about gender issues and identity, ethnicity, accessibility for people from low-income countries, opportunities for early-career scholars - all of this and a lot more is now placed on your shoulders, and you have to respond to it as the you who knew everything about how a conference SHOULD be run would have liked to see it.
Depending on the size of the conference, this will get harder, the feedback and expectations tougher, and rewards for everybody else involved higher, and so the stakes will increase too, and there you are, elbow deep in the stuff you were complaining about before that first seminar.
I still think you should do it. Organise that first seminar. Start learning about how to facilitate the academic growth and development of an academic community. Care enough to act, not just talk. Make the conference you want to see.