-
Notifications
You must be signed in to change notification settings - Fork 2.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How can I include Cobra in my actual project structure ? #910
Comments
@fallais I think I need a bit more clarity on your question. |
@jharshman : Everytime I want to build the two Docker images that correspond to my two microservices (server and collector). I need to build two images, from two Dockerfile. Inside each container, there is an executable, so I need to I would like to go into a continuous integration and build only one container each time I push on my master branch. To do so, I need to package my two microservices into one. I have been hearing about Cobra to acheive that. Then I could add into the Dockerfile only on executable. And play with : Hope it is clear enough. |
@fallais, thanks for the clarification. Yes, cobra is suitable for your use case. However, let's ignore the Dockerfiles for now; it is not relevant for the golang codebase, and it adds unnecessary complexity to the question. I would rephrase the question as: I have two independent golang apps which are built to separate binaries. Now, I want to integrate/merge both of them in a single binary. How can I use cobra to achieve it? I suggest to start by creating a completely new app structure, using cobra's generator: go get -u github.com/spf13/cobra/cobra
mkdir prjdir
cd prjdir
cobra init --pkg-name github.com/user/prjdir
cobra add server
cobra add collector
go mod init github.com/user/prjdir You will get the following structure:
Now, check the codebase, and play with it:
For your specific use case, I would:
You will get:
Now you can merge it with your actual structure:
where:
The main point you need to take into account is that a binary must have a single Should you want to preserve the possibility to build each of the services as independent binaries, I suggest to add a subdir to each of them. I.e.:
In this context,
However, this is up to you. If you don't want to preserve the possibility for independent builds, nor the services being imported by third-party projects, the following structure might be easier to maintain:
The same applies to the Dockerfiles. If services are self-contained, it might be worth creating a single image. However, if they need different dependencies, it might still be useful to create two different images, even though you build a single binary. EDIT If you feel that https://github.com/spf13/cobra/tree/master/cobra falls short of explaining how it can be useful in cases as yours, please do not hesitate to propose a PR! |
Thanks a lot for this really complete answer !! I can't wait to test this ! :) |
@fallais , sounds like you got your questions answered. I'm going to go ahead and close this issue. If you have any more follow up questions, feel free to reach out again! |
@jharshman Yes ! Thanks a lot again :) |
@jharshman : To be honest, I do have a question. If I understand correctly, the whole code that I had it my Thanks a lot |
Just put the code of each microservice in a different function, and call those functions from
This allows users to use your project as a CLI tool or as a library. |
But, the actual main I have is quite huge.. I mean I am declaring the DAOs, the services, the routes... I would have to add all of this code into the Run() of the Cobra command ? If you look above, you will see my actual project structure. One |
For example, this is the package main
import (
"flag"
"net"
"net/http"
"time"
"myProject/cmd/collector/runner"
"myProject/cmd/collector/shared"
"myProject/cmd/collector/system"
"myProject/dao/mongodb"
"myProject/services"
"github.com/prometheus/client_golang/prometheus/promhttp"
"github.com/robfig/cron"
"github.com/sirupsen/logrus"
"github.com/zenazn/goji/graceful"
)
var (
bindAddress = flag.String("bind", ":8000", "Network address used to bind")
logging = flag.String("logging", "info", "Logging level")
configurationFile = flag.String("configuration_file", "configuration.yml", "Configuration file")
databaseHosts = flag.String("db_hosts", "localhost:27017", "Database hosts")
databaseNamespace = flag.String("db_namespace", "xxxxx", "Select the database")
databaseUser = flag.String("db_user", "", "Database user")
databasePassword = flag.String("db_password", "", "Database user password")
runOnStartup = flag.Bool("run_on_startup", false, "Run on startup ?")
recycleOffset = flag.Duration("recycle_offset", 240*time.Hour, "Recycle offset")
)
func init() {
// Parse the flags
flag.Parse()
// Set the logging level
switch *logging {
case "debug":
logrus.SetLevel(logrus.DebugLevel)
case "info":
logrus.SetLevel(logrus.InfoLevel)
case "warn":
logrus.SetLevel(logrus.WarnLevel)
case "error":
logrus.SetLevel(logrus.ErrorLevel)
default:
logrus.SetLevel(logrus.InfoLevel)
}
// Set the TextFormatter
logrus.SetFormatter(&logrus.TextFormatter{
DisableColors: true,
})
logrus.Infoln("xxxxxxx-collector is starting")
}
func main() {
// Share the configuration
shared.DatabaseHosts = *databaseHosts
shared.DatabaseNamespace = *databaseNamespace
shared.DatabaseUser = *databaseUser
shared.DatabasePassword = *databasePassword
// Initialize connection to database
logrus.Infoln("Initializing connection to database")
session, err := system.SetupDatabase()
if err != nil {
logrus.Fatalln("Error when initializing connection to database : ", err)
}
logrus.Infoln("Successfully initialized connection to database")
// Initialize the DAOs
logrus.Infoln("Initializing the DAOs")
indicatorDAO := mongodb.NewIndicatorDAO(session)
logrus.Infoln("Successfully initialized the DAOs")
// Initialize the services
logrus.Infoln("Initializing the services")
indicatorService := services.NewIndicatorService(indicatorDAO)
logrus.Infoln("Successfully initialized the services")
// Setup the providers
providers, err := system.SetupProviders(*configurationFile)
if err != nil {
logrus.Fatalf("Error while setting the providers : %s", err)
}
// Initialize a runner
r := runner.NewRunner(indicatorService, providers, *recycleOffset)
if err != nil {
logrus.Fatalln("Error while initializing the runner", err)
}
// Initialize CRON
c := cron.New()
c.AddFunc("@every 10h", r.Collect)
c.AddFunc("@daily", r.Recycle)
c.Start()
// Handlers
http.Handle("/metrics", promhttp.Handler())
// Initialize the goroutine listening to signals passed to the app
graceful.HandleSignals()
// Pre-graceful shutdown event
graceful.PreHook(func() {
logrus.Infoln("Received a signal, stopping the application")
})
// Post-shutdown event
graceful.PostHook(func() {
// Stop all the taks
c.Stop()
logrus.Infoln("Stopped the application")
})
// Listen to the passed address
logrus.Infoln("Starting the Web server")
listener, err := net.Listen("tcp", *bindAddress)
if err != nil {
logrus.Fatalln("Cannot set up a TCP listener")
}
logrus.Infoln("Successfully started the Web server")
// Run on startup
if *runOnStartup {
logrus.Infoln("Running the job on startup")
r.Recycle()
go r.Collect()
}
// Start the listening
err = graceful.Serve(listener, http.DefaultServeMux)
if err != nil {
logrus.Errorf("Error with the server : %s", err)
}
// Wait until open connections close
graceful.Wait()
} I would need to put all of this code in this : var collectorCmd = &cobra.Command{
Run: func(cmd *cobra.Command, args []string) {
// Do Stuff Here
// Do Stuff Here
// Do Stuff Here
// Do Stuff Here
},
} It is a bit ugly isn't it ? |
If your main function is large, you should look at refactoring it a bit to make it less so. Consider breaking things out into different functions. |
Yes, you have an ugly non-cobra structure and getting the result you want will require some effort on your side. We can spend much time and discuss a lot about it (in the good sense), but I'm afraid that you already have all the information and examples. You just need to sit down, think about it, read this conversation, think about it again, read the docs, and understand it. Then, spend some hours actually implementing the changes. Honestly, I can do little more, except doing it for you, which is not didactic at all.
No. As I said in my previous comment, you just need to call one function from each
As said, you need to convert each Package/lib/file/module with business logic: import (
...
)
var (
// Move flags to `init()` of the corresponding cobra command and use the syntax supported by cobra
)
func init() {
// Move content to the `init()` of the corresponding cobra command
// flag.Parse() NOT required, as it is built in cobra
}
func my_fancy_func_name(args []string) {
// Do Stuff Here
// Do Stuff Here
// Do Stuff Here
// Do Stuff Here
// Do Stuff Here
// Do Stuff Here
// Do Stuff Here
} Corresponding cobra command: var collectorCmd = &cobra.Command{
Run: func(cmd *cobra.Command, args []string) {
my_fancy_func_name(args)
},
} Standalone entrypoint (equivalent to your current func main() {
my_fancy_func_name(os.Args[1:])
} |
Thanks for helping, appreciate. I hope this post will help other people. func collectorSetup(cmd *cobra.Command, args []string) {
// code of my main
// code of my main
// code of my main
} Then do this : var collectorCmd = &cobra.Command{
Run: collectorSetup,
} Right ? |
Exactly. That's even cleaner than what I suggested. EDIT However, note that |
Unless I use it like this : That being said, I think that choosing Cobra mean, going 100% into it. That would make sens to do what I proposed. But I need to think more. EDIT : Or I could use a convient function just for Cobra. Like I am studying the code of Hugo. |
I think you are on the good path now. All of your latest comments/questions are valid and very pertinent. However, I think that you are the only one that can answer/decide. The specific implementation will depend on your needs and preferences; there is no best solution. EDIT After wrapping your head around this, if you feel like proposing a PR to help other users understand these 3-4 different approaches to integrate multiple existing projects into a single cobra project; I'd be happy to review it. Yet, note #959. |
I do not often do this kind of work, how could it be done ? I could add, at the root, an As of today, I fully migrated to Cobra and deployed in production. Quite happy with that. But I am not sure I did the right way, with Logrus for example (logging level setting). Maybe this PR could be a good way to discuss about it ? Actually, I put it in a PersistentPreRunE.. |
Overall, I feel that all the
I think that you can name it
I don't think the example/guide needs to be a fully working example (in the sense that it does anything useful). Precisely, I think that putting application specific logic can be misleading for new users. It's ok if you want to pick names as
I think these are relevant issues, but not to be included in an introduction guide. This is specially so because error handling and log management is not clear enough; hence, we can hardly document something we are not sure about. See #770, #914, #956, #974. You might want to gather the attention of people that participated in those issues in a single place. |
I have this structure (not exhaustive) :
server and collector are my two micro-services. They both have a Dockefile. That means I need to create two images at each build. Boring. That is why I want to migrate to Cobra.
But it seems that my vision of
cmd
interferes with the Cobra's one.How can I do please ? I think about putting a
main.go
at the root ofcmd/
. But how can I link the twocommands
to my two existingmain.go
?Then, the Dockerfile, I assume that it will also be placed at the root of
cmd/
?In fact, problem is that my
cmd
does not contain commands, but the whole micro-services.The text was updated successfully, but these errors were encountered: