{"title": "Optimizing Admission Control while Ensuring Quality of Service in Multimedia Networks via Reinforcement Learning", "book": "Advances in Neural Information Processing Systems", "page_first": 982, "page_last": 988, "abstract": null, "full_text": "Optimizing admission control while ensuring \nquality of service in multimedia networks via \n\nreinforcement learning* \n\nTimothy X Brown t , Hui Tong t , Satinder Singh+ \n\nt Electrical and Computer Engineering \n\n+ Computer Science \nUniversity of Colorado \nBoulder, CO 80309-0425 \n\n{timxb,  tongh,  baveja}@colorado.edu \n\nAbstract \n\nThis  paper  examines  the  application  of  reinforcement  learning  to  a \ntelecommunications networking problem . The problem requires that rev(cid:173)\nenue  be  maximized  while  simultaneously  meeting  a  quality  of service \nconstraint  that  forbids  entry  into  certain  states.  We  present  a  general \nsolution  to  this  multi-criteria  problem  that  is  able  to  earn  significantly \nhigher revenues than alternatives. \n\n1  Introduction \nA  number of researchers have recently explored the application of reinforcement learning \n(RL)  to  resource allocation and admission control problems  in  telecommunications.  e.g., \nchannel allocation in wireless systems, network routing, and admission control in telecom(cid:173)\nmunication  networks  [1,  6,  7,  8].  Telecom  problems are  attractive  applications  for  RL \nresearch because  good, simple to implement, simulation models exist for  them  in  the  en(cid:173)\ngineering  literature  that  are  both  widely  used  and  results  on  which  are  trusted,  because \nthere  are  existing solutions  to  compare  with,  because  small  improvements over existing \nmethods can  lead to  significant savings in  the  long run,  because they have discrete states, \nand  because there are many  potential commercial applications.  However, existing RL ap(cid:173)\nplications  have  ignored  an  issue  of great practical  importance to  telecom  engineers,  that \nof ensuring  quality of service  (QoS)  while  simultaneously optimizing  whatever resource \nallocation performance criterion is  of interest. \n\nThis paper will  focus on admission control for broadband multimedia communication net(cid:173)\nworks.  These networks are  unlike the current internet in  that  voice,  video,  and data calls \narrive and depart over time and,  in  exchange for giving QoS  guarantees to customers, the \nnetwork collects revenue for calls that it accepts into the network.  In this environment, ad(cid:173)\nmission control decides what calls to accept into the network so as to maximize the earned \nrevenue while meeting the QoS guarantees of all  carried customers. \n\n'Timothy Brown and  Hui  Tong were funded  by  NSF CAREER Award  NCR-9624791.  Satinder \n\nSingh was  funded  by  NSF grant IIS-97 1 1753. \n\n\fOptimizing Admission Control via RL \n\n983 \n\nMeeting QoS requires a decision function that decides when adding a new call  will  violate \nQoS  guarantees.  Given the diverse nature of voice,  video, and data traffic, and their often \ncomplex  underlying statistics,  finding  good  QoS  decision  functions  has  been  the  subject \nof intense  research  [2,  5].  Recent results  have emphasized that  robust and  efficient QoS \ndecision functions require on-line adaptive methods [3]. \n\nGiven we have a QoS decision function, deciding which of the heterogeneous arriving calls \nto  accept and  which to  reject  in  order to  maximize revenue can  be  framed  as  a  dynamic \nprogram problem . The rapid growth in the number of states with problem complexity has \nled to reinforcement learning approaches to the problem [6]. \n\nIn this paper we consider the problem of finding a control policy that simultaneously meets \nQoS guarantees and maximizes the network's earned revenue. We show that the straightfor(cid:173)\nward approach of mixing positive rewards for revenue with negative rewards for  violating \nQoS  leads to  sub-optimal  policies.  Ideally we  would  like to find  the  optimal policy  from \nthe  subset of policies that never violate the QoS constraint.  But there is  no a priori  useful \nway  to characterize the  space of policies that don ' t violate the QoS constraint.  We present \na general  approach to  meeting such multicriteria that solves this  problem and  potentially \nmany other applications.  Experiments show  that incorporating QoS  and RL  yield signifi(cid:173)\ncant gains over some alternative heuristics. \n\n2  Problem Description \nThis  section  describes  the  admission  control  problem  model  that  will  be  used.  To  em(cid:173)\nphasize the main  features of the problem, networking issues such as queueing that are not \nessential  have  been  simplified  or eliminated.  It should  be  emphasized that  these aspects \ncan readily be incorporated back into the problem. \n\nWe  focus  on  a  single  network  link.  Users  attempt  to  access  the  link  over  time  and  the \nnetwork  immediately chooses  to  accept or reject the  call.  If accepted,  the  call  generates \ntraffic  in  terms of bandwidth as a function of time.  At a later time, the call  terminates and \ndeparts from  the network.  For each call accepted, the network receives revenue at a fixed \nrate over the duration of the call.  The network measures QoS metrics such as transmission \ndelays  or  packet loss  rates and  compares them  against  the  guarantees  given  to  the  calls. \nThus,  the  problem  is  described  by  the  call  arrival,  traffic,  and  departure  processes;  the \nrevenue  rates;  QoS  metrics;  QoS  constraints;  and  link  model.  The  choices  used  in  this \npaper are given  in  the next paragraph. \n\nCalls  are  divided into  discrete classes  indexed by  i.  The calls are generated via a Poisson \narrival  process  (arrival  rate  Ai) and  exponential holding times (mean holding time 1/ f.Li) . \nWithin a call the bandwidth is  an  ON/OFF process where the traffic is either ON  at rate Ti or \nOFF at  rate  zero  with mean holding times V?N, and  V?FF .  The effective immediate revenue \nare Ct.  The link has a fixed bandwidth B. The total bandwidth used by accepted calls varies \nover time.  The QoS  metric is  the fraction of time that the total  bandwidth exceeds the link \nbandwidth (i.e.  the overload probability, p). The QoS guarantee is  an upper limit, p*. \n\nIn  previous work each call had  a constant bandwidth over time so that the effect on  QoS \nwas  predictable.  Variable  rate  traffic  is  safely  approximated  by  assuming  that  it  always \ntransmits at  its  maximum or peak rate.  Such so-called peak rate  allocation under-utilizes \nthe  network;  in  some cases by  orders of magnitude less  than  what is  possible.  Stochastic \ntraffic rates in  real traffic, the desire for high network utilization/revenue, and the resulting \npotential for QoS violations distinguish the problem in  this paper. \n\n3  Semi-Markov Decision Processes \nAt  any  given  point of time,  the  system  is  in  a  particular configuration, x,  defined  by  the \nnumber of each type of ongoing calls.  At random times a call arrival or a call termination \n\n\f984 \n\nT.  X  Brown,  H.  Tong and S.  Singh \n\nevent, e,  can  occur.  The configuration and  event together determine the  state of the  sys(cid:173)\ntem,  s  =  (x, e).  When  an  event occurs,  the  learner has  to  choose an  action  feasible  for \nthat event.  The choice of action,  the event, and  the configuration deterministically define \nthe  next  configuration  and  the  payoff received  by  the  learner.  Then  after an  interval  the \nnext event occurs,  and  this  cycle repeats.  The  task  of the  learner is  to  determine  a  pol(cid:173)\nicy  that maximizes the discounted sum of payoffs over an  infinite horizon.  Such a system \nconstitutes a finite state, finite  action, semi-Markov decision process (SMDP). \n\n3.1  Multi-criteria Objective \nThe admission control objective is to learn a policy that assigns an accept or reject decision \nto each possible state of the system so as to maximize \n\nJ  =  E  {foOO  ,,/C(t)dt} , \n\nwhere  E{\u00b7}  is  the  expectation  operator,  c(t)  is  the  total  revenue rate  of ongoing calls  at \ntime t, and, E  (0,1)  is a discount factor that makes immediate profit more valuable than \nfuture profit. 1 \n\nIn this paper we restrict the maximization to policies that never enter states that violate QoS \nguarantees.  In  general SMDP,  due to stochastic state transitions,  meeting such constraints \nmay  not  be  possible (e.g.  from  any  state  no matter what actions  are  taken  there  is  a  pos(cid:173)\nsibility  of entering restricted states).  In  this  problem service quality  decreases with  more \ncalls in the system and adding calls is strictly controlIed by the admission controller so that \nmeeting this QoS constraint is possible. \n\n3.2  Q-Iearning \nRL methods solve SMDP problems by  learning good approximations to  the optimal value \nfunction ,  J*,  given  by  the  solution  to  the  Bellman  optimality  equation  which  takes  the \nfollowing form  for the dynamic call  admission problem: \n\nJ*(s) \n\nmax  [E.6.t , s,{c(s,a,flt)+,(L~t)J*(s')}l \naEA(s) \n\n(I) \n\nwhere  A(s)  is  the  set  of actions  available  in  the  current state  s,  flt  is  the  random  time \nuntil the next event, c(s, a,  flt)  is  the effective immediate payoff with the discounting, and \n,(flt) is the effective discount for the next state s' . \n\nWe  learn  an  approximation to  J*  using  Watkin's  Q-learning algorithm.  To  focus  on  the \ndynamics  of this  paper's  problem  and  not on  the  confounding dynamics  of function  ap(cid:173)\nproximation, the problem state space is  kept small enough so that table lookup can be used. \nBellman's equation can be rewritten in Q-values as \n\nJ*(s) \n\nmax  Q*(s,a) \naEA (s) \n\n(2) \n\nCall  Arrival:  When  a  call  arrives.  the  Q-value  of accepting  the  call  and  the  Q-value  of \nrejecting the call is determined. If rejection has the higher value,  we drop the call.  Else,  if \nacceptance has the higher value, we accept the call. \n\nCall Termination:  No action needs to  be taken. \n\nWhatever our decision, we update our value function as follows:  on a transition from  state \ns to s' on action a in time  flt, \n\nQ(s, a) \n\n(1  - 0:)Q(8, a) + 0:  (c(s, a,  flt) + ,(flt)  max  Q(8', b)) \n\nbEArs') \n\n(3) \n\n1 Since we  will compare policies based  on total  reward  rather than discounted sum of reward, we \n\ncan use the Tauberian approximation [4), i.e., r  is chosen to be sufficiently close to  I. \n\n\fOptimizing Admission Control via RL \n\n985 \n\nwhere ex  E  [0, 1]  is  the learning rate. \nIn  order for  Q-Iearning to  perform  well,  all  potentially  important state-action  pairs  (s, a) \nmust be explored.  At each state,  with  probability  E  we apply  an  action  that  will  lead  to  a \nless visited configuration, instead of the action recommended by the Q-value.  However, to \nupdate Q-values we still use the action b recommended by the Q-Iearning. \n\n4  Combining Revenue and Quality of Service \nThe  primary  question  addressed  in  this  paper  is  how  to  combine  the  QoS  constraint \nwith  the  objective  of  maximizing  revenue  within  this  constraint.  Let  p(s, a, ~t) and \nq(s, a, ~t) be  the  revenue and  measured QoS components of the  reward, c(s, a, ~t). Ide(cid:173)\nally  c(s, a, ~t) =  p(s, a, ~t) when  the QoS  constraint is  met and c(s, a, ~t) =  -Large \n(where  -Large is  any  large negative value)  when QoS  is  not met.  If the  QoS  parameters \ncould be accurately measured between each state transition then this approach would be  a \nvalid solution to  the problem.  In  network systems, the QoS metrics contain  a high-degree \nof variability.  For example,  overload probabilities can  be much  smaller than  10- 3  while \nthe  interarrival  periods can  be only a few  ON/OFF cycles  so that except for states with the \nmost egregious QoS  violations, most interarrival periods will have no overloads. \n\nIf the reward is  a general function of revenue and QoS: \n\nc(s, a, ~t) = f(p(s, a, ~t), q(s, a, ~t)), \n\n(4) \n\nsufficient and  necessary  condition for  inducing optimal policy  with  the  QoS  constraint is \ngiven by: \n\nE{J(p(s,a,~t),q(s,a,~t))} = {  ~t~~~a,~t)}  ifE{q(s,a,~t)} <p* \n\notherwise \n\n(5) \n\nFor fe) satisfying this condition, states that violate QoS will be highly penalized and never \nvisited.  The actions for states that are visited will  be based solely on revenue. \nThe Appendix gives a simple example showing that finding  a fO that yields the optimal \npolicy is  unlikely without significant prior knowledge about each state.  Several attempts at \nusing  (4) to combine QoS  and revenue into the reward either violated QoS  or had signifi(cid:173)\ncantly lower reward. \n\nA  straight-forward  alternative  exists  to  meeting  the  multicriteria  formulated  as  follows. \nFor each criteria, j, we estimate a separate set of Q-factors, Qj (s, a).  Each  is  updated via \non-line  Q-Iearning.  These  are  then  combined post facto  at  the  time of decision  via  some \nfunction  Q (.) so that: \n\nQ(s, a)  =  Q( {Qj (s, a)}). \n\nFor example in this paper the two criteria are estimated separately as QP  and Qq  and \n\nQ(s, a)  =  Q(QP(s, a), Qq(s, a))  =  {  ~~~~~)  if Qq(s, a)  < p* \n\notherwise \n\n(6) \n\n(7) \n\nThe structure of this  problem allows  us  to  estimate  Qq  without using (3).  As  stated,  the \nQoS  is  an  intrinsic  property of a state  and  not of future  states  so  it  is  independent of the \npolicy.  This allows us to collect QoS statistics about each state and treat them in a principled \nway  (e.g.  computing confidence  intervals on the  estimates).  Using these QoS  estimates, \nthe  set  of allowable  states  contracts  monotonically over  time  eventually  converging  to  a \nfixed  set of allowable states.  Since the QoS constraint is  guaranteed to reach a fixed  point \nasymptotically, the Q-Iearned policy also approaches a fixed point at the optimal policy via \nstandard  Q-Iearning  proofs.  A  related  scheme  is  analyzed  in  [4]  suggesting  that  similar \ncases will also converge to optimal policies. \n\n\f986 \n\nT.  X  Brown,  H.  Tong and S.  Singh \n\nMany other QoS criteria do depend on the policy and require using (3). A constraint on the \nexpected overload probability with a given policy is  an example. \n\n5  Simulation Results \nThe experiment uses the following model. The total bandwidth is  normalized to  1.0 unit of \ntraffic  per unit time.  The target overflow probability is  p*  =  10- 3 .  Two  source types are \nconsidered with  the  properties shown  in  Table  1.  As  noted  before,  call  holding times  are \nexponential and the arrivals are Poisson. For the first experiment, the ON/OFF holding times \nare exponentially distributed, while for the second experiment, they are Pareto distributed. \nThe Pareto distribution is considered to be a more accurate representation of data traffic. \n\nTable  1:  Experimental parameters \n\nParameter \nON rate, r \n\nMean  ON period, 11l1ON \nMean OFF period, 11l1OFF \nHyperbolic exponent, U  + 1 \n\nCall arrival rate, A \nCall  holding time, II J.L \nImmediate payoff, c \n\nI \n\n0.08 \n\nSource Type \nII \n0.2 \n5 \n45 \n2.12 \n0.2 \n60 \nI \n\n5 \n15 \n2.08 \n0.067 \n\n60 \n5 \n\nIn  the  experiments,  for  each  state-action  pair,  (s , a),  QP(s , a)  is  updated  using  (3).  As \nstated,  in  this  case the  update of Qq(s, a)  does  not need  to  use  (3).  Since  random explo(cid:173)\nration is employed to ensure that all potentially important state-action pairs be tried,  it nat(cid:173)\nurally enables us to  collect statistics that can be used to estimate QoS at these state-action \npairs,  Qq (s, a) .  As  the  number of visits  to each state-action pair increases,  the estimated \nQq(s, a)  becomes more and more accurate and,  with  confidence, we can  gradually elimi(cid:173)\nnate those state-action pairs that will violate QoS requirement. As a consequence, QP(s, a) \nis  updated  in  a  gradually correct subset of state-action space  in  the  sense that QoS  is  met \nfor  any  action  within  this  subspace.  Initial  Q-values  for  RL  are  artificial1y  set  such  that \nQ-Iearning started with the greedy policy (the greedy policy always accepts). \n\nAfter training is completed, we apply a test data set to compare the policy obtained through \nRL with alternative heuristic policies.  The final  QoS  measurements obtained at the end of \nthe  RL training  while learning QoS are  used  for  testing different policies.  To  test  the RL \npolicies,  when  there  is  a  new  call  arrival,  the  algorithm  first  determines  if accepting  this \ncall  will  violate QoS.  If it  will,  the call  is  rejected, else the  action  is  chosen  according to \na  =  argmaxaEA(s) Q(s, a),  where A(s)  = {l=accept, O=reject}.  For the  QoS constraint \nwe  use three cases:  Peak rate allocation; Statistical  multiplexing function  learned on-line. \ndenoted QoS learned; Given statistical  multiplexing function  a priori, denoted QoS given. \nWe  examine six  different cases:  (I) RL: QoS  given;  (2)  RL:  QoS  learned;  (3)  RL:  peak \nrate;  (4) A  heuristic that only accepts calls from  the  most  valuable class,  i.e ..  type  I,  with \ngiven QoS; (5) Greedy:  QoS given; (6) Greedy:  peak rate. \n\nFrom  the  results  shown  in  Fig. \nI ,  it  is  clear  that  simultaneously  doing  Q-Iearning  and \nQoS learning converges correctly to the RL policy obtained by  giving the QoS a priori and \ndoing  standard  Q-Iearning  only.  We  see  significant  gains (about  15%)  due  to  statistical \nmultiplexing:  (6)  vs  (5),  and  (3)  vs  (l).  The gains due to  RL are about 25%:  (6)  vs (3), \nand  (5) vs (2).  Together they yield about 45% increase in  revenue over conservative peak \nrate allocation in this example. It is also clear from the figure that the RL policies perform \nbetter than the heuristic policies.  Fig.  (2) shows the rejection ratios for different policies. \n\n\fOptimizing Admission Control via RL \n\n987 \n\nCompanson 01 dlffer&nt policies. expooentlal OWOFF \n\nl-Greectt  peak ''''. 2-RL  peak rate. a-Greed!  00S grven. '-RL  00S ~arn.d. \"_,,oJ OWOfF \n\nI \n\nJjl / ... O\"\"\n\n\"liDo  g-\n\n.... \n\n__  - 0-\n0 \n\n-\n\n...  --0-----\n\n-0- __  -0 _ \n0 \n\n0 \n\n... \n\n3 \n\n--s-- - g--\n\u2022 \n\n08 \n\n07 \n\n0 6 \n\n09 \n\n0 8 \n\n07 \n\n06 \n\n__  ...\n\n...  _ ...... ......... ... ............ ............ .....................  2_ ... \n\n... \n\no \no \n\nI .AL: OOS given \n2.AL: OOS leamed \n3.AL: peak rate \n4 Greedy. type I only \n5.Greedy\u00b7 QoS given \n6.Greedy- peak rale \n\n05L~,-----,-_--\"-_--:,--'==:c==-====.J \n\no \n\n6 \n10 \nIraining llfT18Sleps(x 106, \n\n8 \n\n12 \n\n14 \n\nFigure) :  Comparison of total  rewards of \nRL  while  learning  QoS,  RL  with  given \nQoS  measurements,  RL  with  peak  rate, \ngreedy  policies  and  peak rate  allocation, \nnormalized  by  the  greedy  total  reward  -\nexponential ON/OFF. \n\nFigure 2:  Comparison of rejection ratios \nfor the policies learned in Fig.  1. \n\nWe  repeat  the  above experiments  with  Pareto  distributed  ON  and  OFF  periods,  using  the \nsame parameters listed in Table  1.  The results are shown in Figs.  3-4. Clearly, the different \nON/OFF distributions yield similar gains for RL. \n\n6  Conclusion \nThis paper shows that a  QoS constraint could be incorporated into a RL solution to  max(cid:173)\nimizing  a  network's  revenue,  using  a  vector  value  Q-learning function.  The formulation \nis  quite general and can be applied to many possible constraints.  The approach,  when ap(cid:173)\nplied  to  a  simple  networking  problem,  increases  revenue  by  up  to  45%.  Future research \nincludes:  using  neural  networks or other function  approximators to  deal  with  more com(cid:173)\nplex  problems for  which lookup tables  are infeasible;  and extending admission control to \nmulti-link routing. \n\n7  Appendix:  Simple One-State Example \nA simple example will  show that a function  with property (5) is  unlikely.  Consider a  link \nthat can accept only one type of call and it can accept no more than one call.  With no actions \npossible when carrying a call  there is  only one state.  Only two rewards are possible, c(R) \nfor  reject and c(A)  for  accept.  To  fix  the  value function  let c(R)  =  0  and let p  and q  be \nthe  random revenues and QoS  experienced.  Analysis of ()  and (2) shows that the accept \naction will  be chosen if and only if E{J(p, q)}  > O. \nIn  this  example, the revenues are random and  possibly  negative (e.g.  if they  are net after \ncost of billing and transport).  The call  should be accepted if E {p}  > 0  and E {q}  < p*. \nTherefore the correct reward function has the property: \n\nE{J(p,q)}  > 0 \n\nif E{p} > 0 and E{q}  < p* \n\n(8) \n\nThe point of the example is  that an  f(\u00b7)  satisfying (8) requires prior knowledge about the \ndistributions of the revenue and the QoS as a function of the state.  Even if it were possible \n\n\f988 \n\nT.  X  Brown, H.  Tong and S.  Singh \n\n1 3 ,-~r-------,-----r-----r---r--r---.----, \n\nCOII'!\"lnson .1_'\" policies. Pareto OIroFF \n\n1-Greedy  peak .... 2-G ... dy' <loS lIlY .... 3- RL  <loS Ioamod. Pare\" ONIOFF \n\nt 2 \n\n11 \n\nI , \n, , \n\nI \nI \n\nI \n\n1  ' \n\n0 9 \n\n08 \n\n0 7 \n\n0 6 \n\nos \n0 \n\n' , -\n\n0 9 \n\n08 \n\n1- - -\n\nRL: QoS leamed \nGteedy: QoS given \n- - ,  Gteedy: peak rate \n\nI 1 \n\n8 \n\n10 \n6 \n'''''flgIImU9pO(x  10~ \n\n12 \n\n\" \n\nII \n\nFigure 3:  Comparison of total rewards of \nRL while learning QoS, greedy policy and \npeak  rate  allocation,  normalized  by  the \ngreedy total reward - Pareto ON/OFF. \n\nFigure 4:  Comparison of rejection ratios \nfor the policies learned in Fig. 3. \n\nfor  this  example,  setting  up constraints such  as  (8)  for  a  real  problem  with  a  huge  state \nspace would be non-trivial because p and q are functions of the many state and action pairs. \n\nReferences \n\n[I]  Boyan,  J.A. ,  Littman,  ML ,  \"Packet routing  in  dynamically  changing  networks:  a \nreinforcement learning  approach,\"  in  Cowan,  J.D. ,  et  al. ,  ed. Advances in  NIPS  6, \nMorgan Kauffman, SF,  1994. pp. 671-678. \n\n[2]  Brown, T.X, \"Adaptive Access Control Applied to Ethernet Data,\" Advances in NIPS \n\n9, ed. M.  Mozer et al., MIT Press,  1997. pp. 932-938. \n\n[3]  Brown, T.X, \"Adaptive Statistical Multiplexing for Broadband Communications,\" In(cid:173)\n\nvited Tutorial Fifth  IFfP Workshop  on Peiformance Modeling  &  Evaluation of ATM \nNetworks, Ilkley, U.K., July,  1997. \n\n[4]  Gabor,  Z.,  Kalmar,  Z.,  Szepesvari, c., \"Multi-criteria Reinforcement Learning,\"  to \nappear in  International Conference on Machine Learning, Madison, WI, July, 1998. \n[5]  Hiramatsu, A., \"ATM Communications Network Control by .Neural Networks,\" IEEE \n\nT  on Neural Networks, v. 1,  n.  1,  pp.  122-130, 1990. \n\n[6]  Marbach, P.,  Mihatsch, 0., Schulte, M ., Tsitsiklis, J.N., \"Reinforcement learning for \ncal1  admission control and routing in  integrated service networks,\"  in  Jordan, M .,  et \naI., ed. Advances in NIPS 10, MIT Press,  1998. \n\n[7]  Nie, J .,  Haykin, S., \"A Q-learning based dynamic channel assignment technique for \n\nmobile communication systems,\" to appear in IEEE T  on Vehicular Technology. \n\n[8]  Singh, S.P., Bertsekas, D.P., \"Reinforcement learning for dynamic channel allocation \nin  cel1ular  telephone  systems,\"  in  Advances in  NIPS  9,  ed.  Mozer,  M .,  et al.,  MIT \nPress,  1997. pp. 974-980. \n\n\f", "award": [], "sourceid": 1599, "authors": [{"given_name": "Timothy", "family_name": "Brown", "institution": null}, {"given_name": "Hui", "family_name": "Tong", "institution": null}, {"given_name": "Satinder", "family_name": "Singh", "institution": null}]}