Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat:remove jprotoc #13145

Merged
merged 20 commits into from
Oct 19, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .licenserc.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,8 @@ header:
- 'dubbo-annotation-processor/src/main/java/org/apache/dubbo/annotation/permit/**'
- 'dubbo-common/src/main/java/org/apache/dubbo/common/logger/helpers/FormattingTuple.java'
- 'dubbo-common/src/main/java/org/apache/dubbo/common/logger/helpers/MessageFormatter.java'
- 'dubbo-maven-plugin/src/main/java/org/apache/dubbo/maven/plugin/protoc/DubboProtocCompilerMojo.java'
- 'dubbo-compiler/src/main/java/org/apache/dubbo/gen/utils/ProtoTypeMap.java'
AlbumenJ marked this conversation as resolved.
Show resolved Hide resolved

comment: on-failure

Expand Down
20 changes: 20 additions & 0 deletions NOTICE
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,23 @@ Copyright 2014 The Netty Project
This product contains code form the t-digest Project:
The code for the t-digest was originally authored by Ted Dunning
Adrien Grand contributed the heart of the AVLTreeDigest (https://github.com/jpountz)

This product contains the following code copied from Maven Protocol Buffers Plugin:
dubbo-maven-plugin/src/main/java/org/apache/dubbo/maven/plugin/protoc/DubboProtocCompilerMojo.java

Maven Protocol Buffers Plugin
=============================
Copyright (c) 2016 Maven Protocol Buffers Plugin Authors. All rights reserved.

This product contains the following code copied from grpc-java-contrib:
dubbo-compiler/src/main/java/org/apache/dubbo/gen/utils/ProtoTypeMap.java
Some portions of this file Copyright (c) 2019, Salesforce.com, Inc. and licensed under the BSD 3-Clause License

grpc-java-contrib
====================
Copyright (c) 2019, Salesforce.com, Inc.
All rights reserved.




137 changes: 137 additions & 0 deletions dubbo-compiler/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
## dubbo-complier

> dubbo-complier supports generating code based on .proto files

### How to use

#### 1.Define Proto file

greeter.proto
```protobuf
syntax = "proto3";

option java_multiple_files = true;
option java_package = "org.apache.dubbo.demo";
option java_outer_classname = "DemoServiceProto";
option objc_class_prefix = "DEMOSRV";

package demoservice;

// The demo service definition.
service DemoService {
rpc SayHello (HelloRequest) returns (HelloReply) {}
}

// The request message containing the user's name.
message HelloRequest {
string name = 1;
}

// The response message containing the greetings
message HelloReply {
string message = 1;
}

```

#### 2.Use dubbo-maven-plugin,rather than ```protobuf-maven-plugin```

now dubbo support his own protoc plugin base on dubbo-maven-plugin

```xml
<plugin>
<groupId>org.apache.dubbo</groupId>
<artifactId>dubbo-maven-plugin</artifactId>
<version>3.3.0</version>
<configuration>
<dubboVersion>3.3.0</dubboVersion>
<dubboGenerateType>dubbo3</dubboGenerateType>
<protocExecutable>protoc</protocExecutable>
<protocArtifact>com.google.protobuf:protoc:${protoc.version}:exe:${os.detected.classifier}</protocArtifact>
</configuration>
</plugin>

```

#### 3.generate file

```java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.apache.dubbo.demo;

import java.util.concurrent.CompletableFuture;
import java.util.concurrent.atomic.AtomicBoolean;

public final class DemoServiceDubbo {
private static final AtomicBoolean registered = new AtomicBoolean();

private static Class<?> init() {
Class<?> clazz = null;
try {
clazz = Class.forName(DemoServiceDubbo.class.getName());
if (registered.compareAndSet(false, true)) {
org.apache.dubbo.common.serialize.protobuf.support.ProtobufUtils.marshaller(
org.apache.dubbo.demo.HelloReply.getDefaultInstance());
org.apache.dubbo.common.serialize.protobuf.support.ProtobufUtils.marshaller(
org.apache.dubbo.demo.HelloRequest.getDefaultInstance());
}
} catch (ClassNotFoundException e) {
// ignore
}
return clazz;
}

private DemoServiceDubbo() {}

public static final String SERVICE_NAME = "org.apache.dubbo.demo.DemoService";

/**
* Code generated for Dubbo
*/
public interface IDemoService extends org.apache.dubbo.rpc.model.DubboStub {

static Class<?> clazz = init();

org.apache.dubbo.demo.HelloReply sayHello(org.apache.dubbo.demo.HelloRequest request);

CompletableFuture<org.apache.dubbo.demo.HelloReply> sayHelloAsync(org.apache.dubbo.demo.HelloRequest request);


}

}

```

#### 4.others

dubbo-maven-plugin protoc mojo supported configurations

| configuration params | isRequired | explain | default | eg |
|:----------------------|------------|------------------------------------------------|------------------------------------------------------------|----------------------------------------------------------------------------|
| dubboVersion | true | dubbo version ,use for find Generator | ${dubbo.version} | 3.3.0 |
| dubboGenerateType | true | dubbo generator type | dubbo3 | grpc |
| protocExecutable | false | protoc executable,you can use local protoc.exe | | protoc |
| protocArtifact | false | download protoc from maven artifact | | com.google.protobuf:protoc:${protoc.version}:exe:${os.detected.classifier} |
| protoSourceDir | true | .proto files dir | ${basedir}/src/main/proto | ./proto |
| outputDir | true | generated file output dir | ${project.build.directory}/generated-sources/protobuf/java | ${basedir}/src/main/java |
| protocPluginDirectory | false | protoc plugin dir | ${project.build.directory}/protoc-plugins | ./target/protoc-plugins |


4 changes: 2 additions & 2 deletions dubbo-compiler/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@

<dependencies>
<dependency>
<groupId>com.salesforce.servicelibs</groupId>
<artifactId>jprotoc</artifactId>
<groupId>com.github.spullara.mustache.java</groupId>
<artifactId>compiler</artifactId>
</dependency>
<dependency>
<groupId>io.grpc</groupId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,29 +16,35 @@
*/
package org.apache.dubbo.gen;

import com.github.mustachejava.DefaultMustacheFactory;
import com.github.mustachejava.Mustache;
import com.github.mustachejava.MustacheFactory;
import com.google.common.base.Charsets;
import com.google.common.base.Preconditions;
import com.google.common.base.Strings;
import com.google.common.html.HtmlEscapers;
import com.google.protobuf.DescriptorProtos.FileDescriptorProto;
import com.google.protobuf.DescriptorProtos.FileOptions;
import com.google.protobuf.DescriptorProtos.MethodDescriptorProto;
import com.google.protobuf.DescriptorProtos.ServiceDescriptorProto;
import com.google.protobuf.DescriptorProtos.SourceCodeInfo.Location;
import com.google.protobuf.compiler.PluginProtos.CodeGeneratorResponse.Feature;
import com.google.protobuf.compiler.PluginProtos;
import com.salesforce.jprotoc.Generator;
import com.salesforce.jprotoc.GeneratorException;
import com.salesforce.jprotoc.ProtoTypeMap;
import org.apache.dubbo.gen.utils.ProtoTypeMap;

import javax.annotation.Nonnull;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.StringWriter;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.stream.Collectors;

public abstract class AbstractGenerator extends Generator {
public abstract class AbstractGenerator {

private static final MustacheFactory MUSTACHE_FACTORY = new DefaultMustacheFactory();
private static final int SERVICE_NUMBER_OF_PATHS = 2;
private static final int METHOD_NUMBER_OF_PATHS = 4;

Expand All @@ -58,10 +64,6 @@ protected String getInterfaceTemplateFileName() {
return getClassPrefix() + getClassSuffix() + "InterfaceStub.mustache";
}

@Override
protected List<Feature> supportedFeatures() {
return Collections.singletonList(Feature.FEATURE_PROTO3_OPTIONAL);
}

private String getServiceJavaDocPrefix() {
return " ";
Expand All @@ -71,9 +73,7 @@ private String getMethodJavaDocPrefix() {
return " ";
}

@Override
public List<PluginProtos.CodeGeneratorResponse.File> generateFiles(
PluginProtos.CodeGeneratorRequest request) throws GeneratorException {
public List<PluginProtos.CodeGeneratorResponse.File> generateFiles(PluginProtos.CodeGeneratorRequest request) {
final ProtoTypeMap typeMap = ProtoTypeMap.of(request.getProtoFileList());

List<FileDescriptorProto> protosToGenerate = request.getProtoFileList().stream()
Expand All @@ -85,12 +85,12 @@ public List<PluginProtos.CodeGeneratorResponse.File> generateFiles(
}

private List<ServiceContext> findServices(List<FileDescriptorProto> protos,
ProtoTypeMap typeMap) {
ProtoTypeMap typeMap) {
List<ServiceContext> contexts = new ArrayList<>();

protos.forEach(fileProto -> {
for (int serviceNumber = 0; serviceNumber < fileProto.getServiceCount();
serviceNumber++) {
serviceNumber++) {
ServiceContext serviceContext = buildServiceContext(
fileProto.getService(serviceNumber),
typeMap,
Expand Down Expand Up @@ -127,7 +127,7 @@ private String extractCommonPackageName(FileDescriptorProto proto) {
}

private ServiceContext buildServiceContext(ServiceDescriptorProto serviceProto,
ProtoTypeMap typeMap, List<Location> locations, int serviceNumber) {
ProtoTypeMap typeMap, List<Location> locations, int serviceNumber) {
ServiceContext serviceContext = new ServiceContext();
serviceContext.fileName =
getClassPrefix() + serviceProto.getName() + getClassSuffix() + ".java";
Expand Down Expand Up @@ -170,7 +170,7 @@ private ServiceContext buildServiceContext(ServiceDescriptorProto serviceProto,
}

private MethodContext buildMethodContext(MethodDescriptorProto methodProto,
ProtoTypeMap typeMap, List<Location> locations, int methodNumber) {
ProtoTypeMap typeMap, List<Location> locations, int methodNumber) {
MethodContext methodContext = new MethodContext();
methodContext.originMethodName = methodProto.getName();
methodContext.methodName = lowerCaseFirst(methodProto.getName());
Expand Down Expand Up @@ -261,6 +261,20 @@ private List<PluginProtos.CodeGeneratorResponse.File> buildFile(ServiceContext c
return files;
}

protected String applyTemplate(@Nonnull String resourcePath, @Nonnull Object generatorContext) {
Preconditions.checkNotNull(resourcePath, "resourcePath");
Preconditions.checkNotNull(generatorContext, "generatorContext");
InputStream resource = MustacheFactory.class.getClassLoader().getResourceAsStream(resourcePath);
if (resource == null) {
throw new RuntimeException("Could not find resource " + resourcePath);
} else {
InputStreamReader resourceReader = new InputStreamReader(resource, Charsets.UTF_8);
Mustache template = MUSTACHE_FACTORY.compile(resourceReader, resourcePath);
return template.execute(new StringWriter(), generatorContext).toString();
}
}


private String absoluteDir(ServiceContext ctx) {
return ctx.packageName.replace('.', '/');
}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dubbo.gen;

import com.google.protobuf.compiler.PluginProtos;

import java.io.IOException;
import java.util.List;


public class DubboGeneratorPlugin {

public static void generate(AbstractGenerator generator) {
try{
PluginProtos.CodeGeneratorRequest request = PluginProtos.CodeGeneratorRequest.parseFrom(System.in);
List<PluginProtos.CodeGeneratorResponse.File> files = generator.generateFiles(request);
PluginProtos.CodeGeneratorResponse.newBuilder().addAllFile(files).setSupportedFeatures(PluginProtos.CodeGeneratorResponse.Feature.FEATURE_PROTO3_OPTIONAL.getNumber()).build().writeTo(System.out);
}catch (Exception e){
try {
PluginProtos.CodeGeneratorResponse.newBuilder().setError(e.getMessage()).build().writeTo(System.out);
} catch (IOException var6) {
exit(e);
}
}catch (Throwable var8) {
exit(var8);
}
}

public static void exit(Throwable e){
e.printStackTrace(System.err);
System.exit(1);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,17 +17,12 @@
package org.apache.dubbo.gen.dubbo;

import org.apache.dubbo.gen.AbstractGenerator;

import com.salesforce.jprotoc.ProtocPlugin;
import org.apache.dubbo.gen.DubboGeneratorPlugin;

public class Dubbo3Generator extends AbstractGenerator {

public static void main(String[] args) {
if (args.length == 0) {
ProtocPlugin.generate(new Dubbo3Generator());
} else {
ProtocPlugin.debug(new Dubbo3Generator(), args[0]);
}
DubboGeneratorPlugin.generate(new Dubbo3Generator());
}

@Override
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,17 +17,13 @@
package org.apache.dubbo.gen.dubbo;

import org.apache.dubbo.gen.AbstractGenerator;
import org.apache.dubbo.gen.DubboGeneratorPlugin;

import com.salesforce.jprotoc.ProtocPlugin;

public class DubboGenerator extends AbstractGenerator {

public static void main(String[] args) {
if (args.length == 0) {
ProtocPlugin.generate(new DubboGenerator());
} else {
ProtocPlugin.debug(new DubboGenerator(), args[0]);
}
DubboGeneratorPlugin.generate(new DubboGenerator());
}

@Override
Expand Down
Loading
Loading