-
Using gRPC from q
gRPC client for q, qrpc was released. Since gRPC is one of the most popular protocol in a large system, q users might have conducted extensive search on the internet to find a solution that can translate your proto files without sacrificing type check, one of the most important feature of motivation of using protobuf. However, gRPC service differs user by user and therefore soon user ended up with implementing ad-hoc q functions.
Here, qrpc helps you implement low level code to encode to/decode from protobuf message and communicate with a gRPC server, and even q code to load them under reasonable namespace. User’s task is to define an environmental variable to point to a directory where proto files are placed.
qrpc]$ export QRPC_PROTO_DIR=path/to/proto/
Then
qrpc]$ cargo build –release
will prepare everything you wanted. You can take the generated shared library (libqrpc.so and q files) to your workspace. The artefacts know what package you are using, and what method to call on what kind of message and what kind of message should be returned. They are tailor-made for you.
I can say here, “Go to the repository right now and try it! You will see how elegant it is!”. However, not everyone has time to explore it without some extent of confidence that it is what you were looking for (yes, life is too busy to devote ourselves to business). Here I will introduce a small example of how a proto file is processed and how its artefacts are used.
We will use following protofile restaurant.proto which can be found in the repository.
syntax="proto3"; package restaurant; import "google/protobuf/empty.proto"; import "q.proto"; // Available menu. enum Menu{ smile = 0; pizza = 1; spaghetti = 2; salad = 3; steak = 4; sushi = 5; hamburger = 6; chips = 7; coke = 8; } // Message representing an order. message Order{ int32 table = 1; repeated Menu items = 2; q.timestamp ordered_time = 3; } // Message representing acceptance. message Acceptance{ bool accepted = 1; string reason = 2; } // Message representing an expense with table ID. message Expense{ int32 table = 1; } // Message representing order history. message History{ q.timestamp time = 1; Menu item = 2; int64 unit = 3; float price = 4; } // Message representing a total due. message Total{ repeated History history = 1; float total = 2; } // Service mocking a restaurant order system. service Restaurant{ // Customer submits an order and a kitchen returns a response. rpc Submit(Order) returns (Acceptance); // Customer finish a meal handing an expense and a restaurant displays a total due // with an order history. rpc Finish(Expense) returns (Total); // Customer forcefully cancels an order. rpc Cancel(Order) returns (google.protobuf.Empty); }Here q.proto is an asset of qrpc which defines derived q types like timestamp, month, symbol etc. This service is mocking a restaurant where customers submit an order, canceling an order and finish their meal and receive a receipt with their order history.
(Note that there is an enum type message Menu. qrpc generates corresponding enum object and cast indices to it when encoding to/decoding from protobuf messages.)
Once you set the QRPC_PROTO_DIR and compile library with cargo, grpc_client_methods.q is generated like below (I omit low level code since it is boring and terrible).
/ * @file grpc_client_methods.q * @overview This is an auto-generated file by qrpc_build crate based on user-specified proto files defining client methods to communicate with a gRPC server. // Source of enum message Menu. .grpc.restaurant.Menu: `smile`pizza`spaghetti`salad`steak`sushi`hamburger`chips`coke; // Load gRPC client method submit in a package restaurant. .grpc.restaurant.submit: `libqrpc 2: (`restaurant_submit; 1); // Load gRPC client method finish in a package restaurant. .grpc.restaurant.finish: `libqrpc 2: (`restaurant_finish; 1); // Load gRPC client method cancel in a package restaurant. .grpc.restaurant.cancel: `libqrpc 2: (`restaurant_cancel; 1)
Enum sources and methods are defined under a namespace composed of a package name (restaurant).
Now you are ready to use gRPC! Assuming we started a gRPC server (example server using the restaurant.proto is provided in the repository as well), we can use the generated code as we will see below. First we have to set endpoint of a gRPC server to connect.
qrpc]$ q/grpc.q q).grpc.set_endpoint[`restaurant; "http://localhost:3160"] "endpoint was set for package: restaurant"
The method .grpc.set_endpoint is a pre-defined method in grpc.q which loads the auto-generated grpc_client_methods.q. This method takes a package name and a server endpoint (The package name is already incorporated in the low-level code and you cannot pass an arbitrary package name).
Once endpoint is set, let’s submit some orders.
q).grpc.restaurant.submit[`table`items`ordered_time!(2i; `.grpc.restaurant.Menu$`pizza`coke`pizza`sushi; .z.p)] accepted| 1 q).grpc.restaurant.submit[`table`items`ordered_time!(2i; `.grpc.restaurant.Menu$`steak`coke`sushi; .z.p)] accepted| 1 q).grpc.restaurant.submit[`table`items`ordered_time!(2i; `.grpc.restaurant.Menu$`steak`steak`chips`coke`spaghetti`hamburger`chips`salad`pizza`sushi; .z.p)] reason| "too many items. must be less than 10"
All messages are expressed with a q dictionary with keys of the field names. We can see that two different messages were returned; one has only bool value populated and another only string. This is reflecting the protobuf specification that default values are not populated.
Next, try to cancel orders and see what happens. In the first attempt we passed a wrong table number and an error was returned from the server. The error message was cast to q error. The second attempt was successful and empty message was returned as a general null.
q).grpc.restaurant.cancel[`table`items`ordered_time!(3i; `.grpc.restaurant.Menu$`sushi`pizza`pizza; .z.p)] 'no order for the table id: 3 [0] .grpc.restaurant.cancel[`table`items`ordered_time!(3i; `.grpc.restaurant.Menu$`sushi`pizza`pizza; .z.p)] ^ q).grpc.restaurant.cancel[`table`items`ordered_time!(2i; `.grpc.restaurant.Menu$`sushi`pizza`pizza; .z.p)]
Fun so far, but now it’s time to go home and enjoy the rest of the day with your beloved one.
q)receipt: .grpc.restaurant.finish[enlist[`table]!enlist 2i] q)receipt history| +`time`item`unit`price!(2022.02.12D11:14:50.217026000 2022.02.12D11:.. total | 23.25e q)receipt `history time item unit price ---------------------------------------------- 2022.02.12D11:14:50.217026000 coke 1 2 2022.02.12D11:15:03.698417000 steak 1 9.25 2022.02.12D11:15:03.698417000 coke 1 2 2022.02.12D11:15:03.698417000 sushi 1 10
Well, we are looking at a nicely formatted message as a table. Remember, dictionaries with same keys are resolved to a table. So this is a result of decoding repeated messages (Of course you can pass a table as repeated message if you want).
That’s it. In this example we used only one package but you can use multiple packages. I hope this article impressed you that using gRPC from q is simple enough with qrpc. You can see more examples in the repository.
-
1 Reply
-
Great example and will be a very useful reference going forward!
Thanks for sharing with such detail
Log in to reply.