rxrpc.txt 33 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858859860861862863864865866867
  1. ======================
  2. RxRPC NETWORK PROTOCOL
  3. ======================
  4. The RxRPC protocol driver provides a reliable two-phase transport on top of UDP
  5. that can be used to perform RxRPC remote operations. This is done over sockets
  6. of AF_RXRPC family, using sendmsg() and recvmsg() with control data to send and
  7. receive data, aborts and errors.
  8. Contents of this document:
  9. (*) Overview.
  10. (*) RxRPC protocol summary.
  11. (*) AF_RXRPC driver model.
  12. (*) Control messages.
  13. (*) Socket options.
  14. (*) Security.
  15. (*) Example client usage.
  16. (*) Example server usage.
  17. (*) AF_RXRPC kernel interface.
  18. ========
  19. OVERVIEW
  20. ========
  21. RxRPC is a two-layer protocol. There is a session layer which provides
  22. reliable virtual connections using UDP over IPv4 (or IPv6) as the transport
  23. layer, but implements a real network protocol; and there's the presentation
  24. layer which renders structured data to binary blobs and back again using XDR
  25. (as does SunRPC):
  26. +-------------+
  27. | Application |
  28. +-------------+
  29. | XDR | Presentation
  30. +-------------+
  31. | RxRPC | Session
  32. +-------------+
  33. | UDP | Transport
  34. +-------------+
  35. AF_RXRPC provides:
  36. (1) Part of an RxRPC facility for both kernel and userspace applications by
  37. making the session part of it a Linux network protocol (AF_RXRPC).
  38. (2) A two-phase protocol. The client transmits a blob (the request) and then
  39. receives a blob (the reply), and the server receives the request and then
  40. transmits the reply.
  41. (3) Retention of the reusable bits of the transport system set up for one call
  42. to speed up subsequent calls.
  43. (4) A secure protocol, using the Linux kernel's key retention facility to
  44. manage security on the client end. The server end must of necessity be
  45. more active in security negotiations.
  46. AF_RXRPC does not provide XDR marshalling/presentation facilities. That is
  47. left to the application. AF_RXRPC only deals in blobs. Even the operation ID
  48. is just the first four bytes of the request blob, and as such is beyond the
  49. kernel's interest.
  50. Sockets of AF_RXRPC family are:
  51. (1) created as type SOCK_DGRAM;
  52. (2) provided with a protocol of the type of underlying transport they're going
  53. to use - currently only PF_INET is supported.
  54. The Andrew File System (AFS) is an example of an application that uses this and
  55. that has both kernel (filesystem) and userspace (utility) components.
  56. ======================
  57. RXRPC PROTOCOL SUMMARY
  58. ======================
  59. An overview of the RxRPC protocol:
  60. (*) RxRPC sits on top of another networking protocol (UDP is the only option
  61. currently), and uses this to provide network transport. UDP ports, for
  62. example, provide transport endpoints.
  63. (*) RxRPC supports multiple virtual "connections" from any given transport
  64. endpoint, thus allowing the endpoints to be shared, even to the same
  65. remote endpoint.
  66. (*) Each connection goes to a particular "service". A connection may not go
  67. to multiple services. A service may be considered the RxRPC equivalent of
  68. a port number. AF_RXRPC permits multiple services to share an endpoint.
  69. (*) Client-originating packets are marked, thus a transport endpoint can be
  70. shared between client and server connections (connections have a
  71. direction).
  72. (*) Up to a billion connections may be supported concurrently between one
  73. local transport endpoint and one service on one remote endpoint. An RxRPC
  74. connection is described by seven numbers:
  75. Local address }
  76. Local port } Transport (UDP) address
  77. Remote address }
  78. Remote port }
  79. Direction
  80. Connection ID
  81. Service ID
  82. (*) Each RxRPC operation is a "call". A connection may make up to four
  83. billion calls, but only up to four calls may be in progress on a
  84. connection at any one time.
  85. (*) Calls are two-phase and asymmetric: the client sends its request data,
  86. which the service receives; then the service transmits the reply data
  87. which the client receives.
  88. (*) The data blobs are of indefinite size, the end of a phase is marked with a
  89. flag in the packet. The number of packets of data making up one blob may
  90. not exceed 4 billion, however, as this would cause the sequence number to
  91. wrap.
  92. (*) The first four bytes of the request data are the service operation ID.
  93. (*) Security is negotiated on a per-connection basis. The connection is
  94. initiated by the first data packet on it arriving. If security is
  95. requested, the server then issues a "challenge" and then the client
  96. replies with a "response". If the response is successful, the security is
  97. set for the lifetime of that connection, and all subsequent calls made
  98. upon it use that same security. In the event that the server lets a
  99. connection lapse before the client, the security will be renegotiated if
  100. the client uses the connection again.
  101. (*) Calls use ACK packets to handle reliability. Data packets are also
  102. explicitly sequenced per call.
  103. (*) There are two types of positive acknowledgement: hard-ACKs and soft-ACKs.
  104. A hard-ACK indicates to the far side that all the data received to a point
  105. has been received and processed; a soft-ACK indicates that the data has
  106. been received but may yet be discarded and re-requested. The sender may
  107. not discard any transmittable packets until they've been hard-ACK'd.
  108. (*) Reception of a reply data packet implicitly hard-ACK's all the data
  109. packets that make up the request.
  110. (*) An call is complete when the request has been sent, the reply has been
  111. received and the final hard-ACK on the last packet of the reply has
  112. reached the server.
  113. (*) An call may be aborted by either end at any time up to its completion.
  114. =====================
  115. AF_RXRPC DRIVER MODEL
  116. =====================
  117. About the AF_RXRPC driver:
  118. (*) The AF_RXRPC protocol transparently uses internal sockets of the transport
  119. protocol to represent transport endpoints.
  120. (*) AF_RXRPC sockets map onto RxRPC connection bundles. Actual RxRPC
  121. connections are handled transparently. One client socket may be used to
  122. make multiple simultaneous calls to the same service. One server socket
  123. may handle calls from many clients.
  124. (*) Additional parallel client connections will be initiated to support extra
  125. concurrent calls, up to a tunable limit.
  126. (*) Each connection is retained for a certain amount of time [tunable] after
  127. the last call currently using it has completed in case a new call is made
  128. that could reuse it.
  129. (*) Each internal UDP socket is retained [tunable] for a certain amount of
  130. time [tunable] after the last connection using it discarded, in case a new
  131. connection is made that could use it.
  132. (*) A client-side connection is only shared between calls if they have have
  133. the same key struct describing their security (and assuming the calls
  134. would otherwise share the connection). Non-secured calls would also be
  135. able to share connections with each other.
  136. (*) A server-side connection is shared if the client says it is.
  137. (*) ACK'ing is handled by the protocol driver automatically, including ping
  138. replying.
  139. (*) SO_KEEPALIVE automatically pings the other side to keep the connection
  140. alive [TODO].
  141. (*) If an ICMP error is received, all calls affected by that error will be
  142. aborted with an appropriate network error passed through recvmsg().
  143. Interaction with the user of the RxRPC socket:
  144. (*) A socket is made into a server socket by binding an address with a
  145. non-zero service ID.
  146. (*) In the client, sending a request is achieved with one or more sendmsgs,
  147. followed by the reply being received with one or more recvmsgs.
  148. (*) The first sendmsg for a request to be sent from a client contains a tag to
  149. be used in all other sendmsgs or recvmsgs associated with that call. The
  150. tag is carried in the control data.
  151. (*) connect() is used to supply a default destination address for a client
  152. socket. This may be overridden by supplying an alternate address to the
  153. first sendmsg() of a call (struct msghdr::msg_name).
  154. (*) If connect() is called on an unbound client, a random local port will
  155. bound before the operation takes place.
  156. (*) A server socket may also be used to make client calls. To do this, the
  157. first sendmsg() of the call must specify the target address. The server's
  158. transport endpoint is used to send the packets.
  159. (*) Once the application has received the last message associated with a call,
  160. the tag is guaranteed not to be seen again, and so it can be used to pin
  161. client resources. A new call can then be initiated with the same tag
  162. without fear of interference.
  163. (*) In the server, a request is received with one or more recvmsgs, then the
  164. the reply is transmitted with one or more sendmsgs, and then the final ACK
  165. is received with a last recvmsg.
  166. (*) When sending data for a call, sendmsg is given MSG_MORE if there's more
  167. data to come on that call.
  168. (*) When receiving data for a call, recvmsg flags MSG_MORE if there's more
  169. data to come for that call.
  170. (*) When receiving data or messages for a call, MSG_EOR is flagged by recvmsg
  171. to indicate the terminal message for that call.
  172. (*) A call may be aborted by adding an abort control message to the control
  173. data. Issuing an abort terminates the kernel's use of that call's tag.
  174. Any messages waiting in the receive queue for that call will be discarded.
  175. (*) Aborts, busy notifications and challenge packets are delivered by recvmsg,
  176. and control data messages will be set to indicate the context. Receiving
  177. an abort or a busy message terminates the kernel's use of that call's tag.
  178. (*) The control data part of the msghdr struct is used for a number of things:
  179. (*) The tag of the intended or affected call.
  180. (*) Sending or receiving errors, aborts and busy notifications.
  181. (*) Notifications of incoming calls.
  182. (*) Sending debug requests and receiving debug replies [TODO].
  183. (*) When the kernel has received and set up an incoming call, it sends a
  184. message to server application to let it know there's a new call awaiting
  185. its acceptance [recvmsg reports a special control message]. The server
  186. application then uses sendmsg to assign a tag to the new call. Once that
  187. is done, the first part of the request data will be delivered by recvmsg.
  188. (*) The server application has to provide the server socket with a keyring of
  189. secret keys corresponding to the security types it permits. When a secure
  190. connection is being set up, the kernel looks up the appropriate secret key
  191. in the keyring and then sends a challenge packet to the client and
  192. receives a response packet. The kernel then checks the authorisation of
  193. the packet and either aborts the connection or sets up the security.
  194. (*) The name of the key a client will use to secure its communications is
  195. nominated by a socket option.
  196. Notes on recvmsg:
  197. (*) If there's a sequence of data messages belonging to a particular call on
  198. the receive queue, then recvmsg will keep working through them until:
  199. (a) it meets the end of that call's received data,
  200. (b) it meets a non-data message,
  201. (c) it meets a message belonging to a different call, or
  202. (d) it fills the user buffer.
  203. If recvmsg is called in blocking mode, it will keep sleeping, awaiting the
  204. reception of further data, until one of the above four conditions is met.
  205. (2) MSG_PEEK operates similarly, but will return immediately if it has put any
  206. data in the buffer rather than sleeping until it can fill the buffer.
  207. (3) If a data message is only partially consumed in filling a user buffer,
  208. then the remainder of that message will be left on the front of the queue
  209. for the next taker. MSG_TRUNC will never be flagged.
  210. (4) If there is more data to be had on a call (it hasn't copied the last byte
  211. of the last data message in that phase yet), then MSG_MORE will be
  212. flagged.
  213. ================
  214. CONTROL MESSAGES
  215. ================
  216. AF_RXRPC makes use of control messages in sendmsg() and recvmsg() to multiplex
  217. calls, to invoke certain actions and to report certain conditions. These are:
  218. MESSAGE ID SRT DATA MEANING
  219. ======================= === =========== ===============================
  220. RXRPC_USER_CALL_ID sr- User ID App's call specifier
  221. RXRPC_ABORT srt Abort code Abort code to issue/received
  222. RXRPC_ACK -rt n/a Final ACK received
  223. RXRPC_NET_ERROR -rt error num Network error on call
  224. RXRPC_BUSY -rt n/a Call rejected (server busy)
  225. RXRPC_LOCAL_ERROR -rt error num Local error encountered
  226. RXRPC_NEW_CALL -r- n/a New call received
  227. RXRPC_ACCEPT s-- n/a Accept new call
  228. (SRT = usable in Sendmsg / delivered by Recvmsg / Terminal message)
  229. (*) RXRPC_USER_CALL_ID
  230. This is used to indicate the application's call ID. It's an unsigned long
  231. that the app specifies in the client by attaching it to the first data
  232. message or in the server by passing it in association with an RXRPC_ACCEPT
  233. message. recvmsg() passes it in conjunction with all messages except
  234. those of the RXRPC_NEW_CALL message.
  235. (*) RXRPC_ABORT
  236. This is can be used by an application to abort a call by passing it to
  237. sendmsg, or it can be delivered by recvmsg to indicate a remote abort was
  238. received. Either way, it must be associated with an RXRPC_USER_CALL_ID to
  239. specify the call affected. If an abort is being sent, then error EBADSLT
  240. will be returned if there is no call with that user ID.
  241. (*) RXRPC_ACK
  242. This is delivered to a server application to indicate that the final ACK
  243. of a call was received from the client. It will be associated with an
  244. RXRPC_USER_CALL_ID to indicate the call that's now complete.
  245. (*) RXRPC_NET_ERROR
  246. This is delivered to an application to indicate that an ICMP error message
  247. was encountered in the process of trying to talk to the peer. An
  248. errno-class integer value will be included in the control message data
  249. indicating the problem, and an RXRPC_USER_CALL_ID will indicate the call
  250. affected.
  251. (*) RXRPC_BUSY
  252. This is delivered to a client application to indicate that a call was
  253. rejected by the server due to the server being busy. It will be
  254. associated with an RXRPC_USER_CALL_ID to indicate the rejected call.
  255. (*) RXRPC_LOCAL_ERROR
  256. This is delivered to an application to indicate that a local error was
  257. encountered and that a call has been aborted because of it. An
  258. errno-class integer value will be included in the control message data
  259. indicating the problem, and an RXRPC_USER_CALL_ID will indicate the call
  260. affected.
  261. (*) RXRPC_NEW_CALL
  262. This is delivered to indicate to a server application that a new call has
  263. arrived and is awaiting acceptance. No user ID is associated with this,
  264. as a user ID must subsequently be assigned by doing an RXRPC_ACCEPT.
  265. (*) RXRPC_ACCEPT
  266. This is used by a server application to attempt to accept a call and
  267. assign it a user ID. It should be associated with an RXRPC_USER_CALL_ID
  268. to indicate the user ID to be assigned. If there is no call to be
  269. accepted (it may have timed out, been aborted, etc.), then sendmsg will
  270. return error ENODATA. If the user ID is already in use by another call,
  271. then error EBADSLT will be returned.
  272. ==============
  273. SOCKET OPTIONS
  274. ==============
  275. AF_RXRPC sockets support a few socket options at the SOL_RXRPC level:
  276. (*) RXRPC_SECURITY_KEY
  277. This is used to specify the description of the key to be used. The key is
  278. extracted from the calling process's keyrings with request_key() and
  279. should be of "rxrpc" type.
  280. The optval pointer points to the description string, and optlen indicates
  281. how long the string is, without the NUL terminator.
  282. (*) RXRPC_SECURITY_KEYRING
  283. Similar to above but specifies a keyring of server secret keys to use (key
  284. type "keyring"). See the "Security" section.
  285. (*) RXRPC_EXCLUSIVE_CONNECTION
  286. This is used to request that new connections should be used for each call
  287. made subsequently on this socket. optval should be NULL and optlen 0.
  288. (*) RXRPC_MIN_SECURITY_LEVEL
  289. This is used to specify the minimum security level required for calls on
  290. this socket. optval must point to an int containing one of the following
  291. values:
  292. (a) RXRPC_SECURITY_PLAIN
  293. Encrypted checksum only.
  294. (b) RXRPC_SECURITY_AUTH
  295. Encrypted checksum plus packet padded and first eight bytes of packet
  296. encrypted - which includes the actual packet length.
  297. (c) RXRPC_SECURITY_ENCRYPTED
  298. Encrypted checksum plus entire packet padded and encrypted, including
  299. actual packet length.
  300. ========
  301. SECURITY
  302. ========
  303. Currently, only the kerberos 4 equivalent protocol has been implemented
  304. (security index 2 - rxkad). This requires the rxkad module to be loaded and,
  305. on the client, tickets of the appropriate type to be obtained from the AFS
  306. kaserver or the kerberos server and installed as "rxrpc" type keys. This is
  307. normally done using the klog program. An example simple klog program can be
  308. found at:
  309. http://people.redhat.com/~dhowells/rxrpc/klog.c
  310. The payload provided to add_key() on the client should be of the following
  311. form:
  312. struct rxrpc_key_sec2_v1 {
  313. uint16_t security_index; /* 2 */
  314. uint16_t ticket_length; /* length of ticket[] */
  315. uint32_t expiry; /* time at which expires */
  316. uint8_t kvno; /* key version number */
  317. uint8_t __pad[3];
  318. uint8_t session_key[8]; /* DES session key */
  319. uint8_t ticket[0]; /* the encrypted ticket */
  320. };
  321. Where the ticket blob is just appended to the above structure.
  322. For the server, keys of type "rxrpc_s" must be made available to the server.
  323. They have a description of "<serviceID>:<securityIndex>" (eg: "52:2" for an
  324. rxkad key for the AFS VL service). When such a key is created, it should be
  325. given the server's secret key as the instantiation data (see the example
  326. below).
  327. add_key("rxrpc_s", "52:2", secret_key, 8, keyring);
  328. A keyring is passed to the server socket by naming it in a sockopt. The server
  329. socket then looks the server secret keys up in this keyring when secure
  330. incoming connections are made. This can be seen in an example program that can
  331. be found at:
  332. http://people.redhat.com/~dhowells/rxrpc/listen.c
  333. ====================
  334. EXAMPLE CLIENT USAGE
  335. ====================
  336. A client would issue an operation by:
  337. (1) An RxRPC socket is set up by:
  338. client = socket(AF_RXRPC, SOCK_DGRAM, PF_INET);
  339. Where the third parameter indicates the protocol family of the transport
  340. socket used - usually IPv4 but it can also be IPv6 [TODO].
  341. (2) A local address can optionally be bound:
  342. struct sockaddr_rxrpc srx = {
  343. .srx_family = AF_RXRPC,
  344. .srx_service = 0, /* we're a client */
  345. .transport_type = SOCK_DGRAM, /* type of transport socket */
  346. .transport.sin_family = AF_INET,
  347. .transport.sin_port = htons(7000), /* AFS callback */
  348. .transport.sin_address = 0, /* all local interfaces */
  349. };
  350. bind(client, &srx, sizeof(srx));
  351. This specifies the local UDP port to be used. If not given, a random
  352. non-privileged port will be used. A UDP port may be shared between
  353. several unrelated RxRPC sockets. Security is handled on a basis of
  354. per-RxRPC virtual connection.
  355. (3) The security is set:
  356. const char *key = "AFS:cambridge.redhat.com";
  357. setsockopt(client, SOL_RXRPC, RXRPC_SECURITY_KEY, key, strlen(key));
  358. This issues a request_key() to get the key representing the security
  359. context. The minimum security level can be set:
  360. unsigned int sec = RXRPC_SECURITY_ENCRYPTED;
  361. setsockopt(client, SOL_RXRPC, RXRPC_MIN_SECURITY_LEVEL,
  362. &sec, sizeof(sec));
  363. (4) The server to be contacted can then be specified (alternatively this can
  364. be done through sendmsg):
  365. struct sockaddr_rxrpc srx = {
  366. .srx_family = AF_RXRPC,
  367. .srx_service = VL_SERVICE_ID,
  368. .transport_type = SOCK_DGRAM, /* type of transport socket */
  369. .transport.sin_family = AF_INET,
  370. .transport.sin_port = htons(7005), /* AFS volume manager */
  371. .transport.sin_address = ...,
  372. };
  373. connect(client, &srx, sizeof(srx));
  374. (5) The request data should then be posted to the server socket using a series
  375. of sendmsg() calls, each with the following control message attached:
  376. RXRPC_USER_CALL_ID - specifies the user ID for this call
  377. MSG_MORE should be set in msghdr::msg_flags on all but the last part of
  378. the request. Multiple requests may be made simultaneously.
  379. If a call is intended to go to a destination other than the default
  380. specified through connect(), then msghdr::msg_name should be set on the
  381. first request message of that call.
  382. (6) The reply data will then be posted to the server socket for recvmsg() to
  383. pick up. MSG_MORE will be flagged by recvmsg() if there's more reply data
  384. for a particular call to be read. MSG_EOR will be set on the terminal
  385. read for a call.
  386. All data will be delivered with the following control message attached:
  387. RXRPC_USER_CALL_ID - specifies the user ID for this call
  388. If an abort or error occurred, this will be returned in the control data
  389. buffer instead, and MSG_EOR will be flagged to indicate the end of that
  390. call.
  391. ====================
  392. EXAMPLE SERVER USAGE
  393. ====================
  394. A server would be set up to accept operations in the following manner:
  395. (1) An RxRPC socket is created by:
  396. server = socket(AF_RXRPC, SOCK_DGRAM, PF_INET);
  397. Where the third parameter indicates the address type of the transport
  398. socket used - usually IPv4.
  399. (2) Security is set up if desired by giving the socket a keyring with server
  400. secret keys in it:
  401. keyring = add_key("keyring", "AFSkeys", NULL, 0,
  402. KEY_SPEC_PROCESS_KEYRING);
  403. const char secret_key[8] = {
  404. 0xa7, 0x83, 0x8a, 0xcb, 0xc7, 0x83, 0xec, 0x94 };
  405. add_key("rxrpc_s", "52:2", secret_key, 8, keyring);
  406. setsockopt(server, SOL_RXRPC, RXRPC_SECURITY_KEYRING, "AFSkeys", 7);
  407. The keyring can be manipulated after it has been given to the socket. This
  408. permits the server to add more keys, replace keys, etc. whilst it is live.
  409. (2) A local address must then be bound:
  410. struct sockaddr_rxrpc srx = {
  411. .srx_family = AF_RXRPC,
  412. .srx_service = VL_SERVICE_ID, /* RxRPC service ID */
  413. .transport_type = SOCK_DGRAM, /* type of transport socket */
  414. .transport.sin_family = AF_INET,
  415. .transport.sin_port = htons(7000), /* AFS callback */
  416. .transport.sin_address = 0, /* all local interfaces */
  417. };
  418. bind(server, &srx, sizeof(srx));
  419. (3) The server is then set to listen out for incoming calls:
  420. listen(server, 100);
  421. (4) The kernel notifies the server of pending incoming connections by sending
  422. it a message for each. This is received with recvmsg() on the server
  423. socket. It has no data, and has a single dataless control message
  424. attached:
  425. RXRPC_NEW_CALL
  426. The address that can be passed back by recvmsg() at this point should be
  427. ignored since the call for which the message was posted may have gone by
  428. the time it is accepted - in which case the first call still on the queue
  429. will be accepted.
  430. (5) The server then accepts the new call by issuing a sendmsg() with two
  431. pieces of control data and no actual data:
  432. RXRPC_ACCEPT - indicate connection acceptance
  433. RXRPC_USER_CALL_ID - specify user ID for this call
  434. (6) The first request data packet will then be posted to the server socket for
  435. recvmsg() to pick up. At that point, the RxRPC address for the call can
  436. be read from the address fields in the msghdr struct.
  437. Subsequent request data will be posted to the server socket for recvmsg()
  438. to collect as it arrives. All but the last piece of the request data will
  439. be delivered with MSG_MORE flagged.
  440. All data will be delivered with the following control message attached:
  441. RXRPC_USER_CALL_ID - specifies the user ID for this call
  442. (8) The reply data should then be posted to the server socket using a series
  443. of sendmsg() calls, each with the following control messages attached:
  444. RXRPC_USER_CALL_ID - specifies the user ID for this call
  445. MSG_MORE should be set in msghdr::msg_flags on all but the last message
  446. for a particular call.
  447. (9) The final ACK from the client will be posted for retrieval by recvmsg()
  448. when it is received. It will take the form of a dataless message with two
  449. control messages attached:
  450. RXRPC_USER_CALL_ID - specifies the user ID for this call
  451. RXRPC_ACK - indicates final ACK (no data)
  452. MSG_EOR will be flagged to indicate that this is the final message for
  453. this call.
  454. (10) Up to the point the final packet of reply data is sent, the call can be
  455. aborted by calling sendmsg() with a dataless message with the following
  456. control messages attached:
  457. RXRPC_USER_CALL_ID - specifies the user ID for this call
  458. RXRPC_ABORT - indicates abort code (4 byte data)
  459. Any packets waiting in the socket's receive queue will be discarded if
  460. this is issued.
  461. Note that all the communications for a particular service take place through
  462. the one server socket, using control messages on sendmsg() and recvmsg() to
  463. determine the call affected.
  464. =========================
  465. AF_RXRPC KERNEL INTERFACE
  466. =========================
  467. The AF_RXRPC module also provides an interface for use by in-kernel utilities
  468. such as the AFS filesystem. This permits such a utility to:
  469. (1) Use different keys directly on individual client calls on one socket
  470. rather than having to open a whole slew of sockets, one for each key it
  471. might want to use.
  472. (2) Avoid having RxRPC call request_key() at the point of issue of a call or
  473. opening of a socket. Instead the utility is responsible for requesting a
  474. key at the appropriate point. AFS, for instance, would do this during VFS
  475. operations such as open() or unlink(). The key is then handed through
  476. when the call is initiated.
  477. (3) Request the use of something other than GFP_KERNEL to allocate memory.
  478. (4) Avoid the overhead of using the recvmsg() call. RxRPC messages can be
  479. intercepted before they get put into the socket Rx queue and the socket
  480. buffers manipulated directly.
  481. To use the RxRPC facility, a kernel utility must still open an AF_RXRPC socket,
  482. bind an address as appropriate and listen if it's to be a server socket, but
  483. then it passes this to the kernel interface functions.
  484. The kernel interface functions are as follows:
  485. (*) Begin a new client call.
  486. struct rxrpc_call *
  487. rxrpc_kernel_begin_call(struct socket *sock,
  488. struct sockaddr_rxrpc *srx,
  489. struct key *key,
  490. unsigned long user_call_ID,
  491. gfp_t gfp);
  492. This allocates the infrastructure to make a new RxRPC call and assigns
  493. call and connection numbers. The call will be made on the UDP port that
  494. the socket is bound to. The call will go to the destination address of a
  495. connected client socket unless an alternative is supplied (srx is
  496. non-NULL).
  497. If a key is supplied then this will be used to secure the call instead of
  498. the key bound to the socket with the RXRPC_SECURITY_KEY sockopt. Calls
  499. secured in this way will still share connections if at all possible.
  500. The user_call_ID is equivalent to that supplied to sendmsg() in the
  501. control data buffer. It is entirely feasible to use this to point to a
  502. kernel data structure.
  503. If this function is successful, an opaque reference to the RxRPC call is
  504. returned. The caller now holds a reference on this and it must be
  505. properly ended.
  506. (*) End a client call.
  507. void rxrpc_kernel_end_call(struct rxrpc_call *call);
  508. This is used to end a previously begun call. The user_call_ID is expunged
  509. from AF_RXRPC's knowledge and will not be seen again in association with
  510. the specified call.
  511. (*) Send data through a call.
  512. int rxrpc_kernel_send_data(struct rxrpc_call *call, struct msghdr *msg,
  513. size_t len);
  514. This is used to supply either the request part of a client call or the
  515. reply part of a server call. msg.msg_iovlen and msg.msg_iov specify the
  516. data buffers to be used. msg_iov may not be NULL and must point
  517. exclusively to in-kernel virtual addresses. msg.msg_flags may be given
  518. MSG_MORE if there will be subsequent data sends for this call.
  519. The msg must not specify a destination address, control data or any flags
  520. other than MSG_MORE. len is the total amount of data to transmit.
  521. (*) Abort a call.
  522. void rxrpc_kernel_abort_call(struct rxrpc_call *call, u32 abort_code);
  523. This is used to abort a call if it's still in an abortable state. The
  524. abort code specified will be placed in the ABORT message sent.
  525. (*) Intercept received RxRPC messages.
  526. typedef void (*rxrpc_interceptor_t)(struct sock *sk,
  527. unsigned long user_call_ID,
  528. struct sk_buff *skb);
  529. void
  530. rxrpc_kernel_intercept_rx_messages(struct socket *sock,
  531. rxrpc_interceptor_t interceptor);
  532. This installs an interceptor function on the specified AF_RXRPC socket.
  533. All messages that would otherwise wind up in the socket's Rx queue are
  534. then diverted to this function. Note that care must be taken to process
  535. the messages in the right order to maintain DATA message sequentiality.
  536. The interceptor function itself is provided with the address of the socket
  537. and handling the incoming message, the ID assigned by the kernel utility
  538. to the call and the socket buffer containing the message.
  539. The skb->mark field indicates the type of message:
  540. MARK MEANING
  541. =============================== =======================================
  542. RXRPC_SKB_MARK_DATA Data message
  543. RXRPC_SKB_MARK_FINAL_ACK Final ACK received for an incoming call
  544. RXRPC_SKB_MARK_BUSY Client call rejected as server busy
  545. RXRPC_SKB_MARK_REMOTE_ABORT Call aborted by peer
  546. RXRPC_SKB_MARK_NET_ERROR Network error detected
  547. RXRPC_SKB_MARK_LOCAL_ERROR Local error encountered
  548. RXRPC_SKB_MARK_NEW_CALL New incoming call awaiting acceptance
  549. The remote abort message can be probed with rxrpc_kernel_get_abort_code().
  550. The two error messages can be probed with rxrpc_kernel_get_error_number().
  551. A new call can be accepted with rxrpc_kernel_accept_call().
  552. Data messages can have their contents extracted with the usual bunch of
  553. socket buffer manipulation functions. A data message can be determined to
  554. be the last one in a sequence with rxrpc_kernel_is_data_last(). When a
  555. data message has been used up, rxrpc_kernel_data_delivered() should be
  556. called on it..
  557. Non-data messages should be handled to rxrpc_kernel_free_skb() to dispose
  558. of. It is possible to get extra refs on all types of message for later
  559. freeing, but this may pin the state of a call until the message is finally
  560. freed.
  561. (*) Accept an incoming call.
  562. struct rxrpc_call *
  563. rxrpc_kernel_accept_call(struct socket *sock,
  564. unsigned long user_call_ID);
  565. This is used to accept an incoming call and to assign it a call ID. This
  566. function is similar to rxrpc_kernel_begin_call() and calls accepted must
  567. be ended in the same way.
  568. If this function is successful, an opaque reference to the RxRPC call is
  569. returned. The caller now holds a reference on this and it must be
  570. properly ended.
  571. (*) Reject an incoming call.
  572. int rxrpc_kernel_reject_call(struct socket *sock);
  573. This is used to reject the first incoming call on the socket's queue with
  574. a BUSY message. -ENODATA is returned if there were no incoming calls.
  575. Other errors may be returned if the call had been aborted (-ECONNABORTED)
  576. or had timed out (-ETIME).
  577. (*) Record the delivery of a data message and free it.
  578. void rxrpc_kernel_data_delivered(struct sk_buff *skb);
  579. This is used to record a data message as having been delivered and to
  580. update the ACK state for the call. The socket buffer will be freed.
  581. (*) Free a message.
  582. void rxrpc_kernel_free_skb(struct sk_buff *skb);
  583. This is used to free a non-DATA socket buffer intercepted from an AF_RXRPC
  584. socket.
  585. (*) Determine if a data message is the last one on a call.
  586. bool rxrpc_kernel_is_data_last(struct sk_buff *skb);
  587. This is used to determine if a socket buffer holds the last data message
  588. to be received for a call (true will be returned if it does, false
  589. if not).
  590. The data message will be part of the reply on a client call and the
  591. request on an incoming call. In the latter case there will be more
  592. messages, but in the former case there will not.
  593. (*) Get the abort code from an abort message.
  594. u32 rxrpc_kernel_get_abort_code(struct sk_buff *skb);
  595. This is used to extract the abort code from a remote abort message.
  596. (*) Get the error number from a local or network error message.
  597. int rxrpc_kernel_get_error_number(struct sk_buff *skb);
  598. This is used to extract the error number from a message indicating either
  599. a local error occurred or a network error occurred.
  600. (*) Allocate a null key for doing anonymous security.
  601. struct key *rxrpc_get_null_key(const char *keyname);
  602. This is used to allocate a null RxRPC key that can be used to indicate
  603. anonymous security for a particular domain.