#w_1

[dynamitekansai]

hanadayo0903: 3D撮影なう！ #w_1 #pwACE #QLQL

[nostalgebraist]

OK yeah, that thing I was talking to @raginrayguns about is way simpler than I thought

The Kelly criterion maximizes the rate of exponential growth, which is just

log(final / initial)

up to a constant.

Like if you have w(t) = exp(rate * t) , and you end at t=T, then

rate = 1/T log(w(T) / w(0))

and T is a constant.

So the Kelly criterion really is nothing but maximizing log wealth, only phrased equivalently as "maximizing exponential growth rate."

And this phrasing is confusing, because "maximizing exponential growth rate" sounds sort of generically good. Like why wouldn't you want that?

But the equivalence goes both ways: it's the same thing as maximizing log wealth, and it's easy to see you may not want that.

----

I made a mistake in my original post about geometric averages -- I linked to a twitter thread about the Kelly criterion, and a blog post by the same person, as if they were making the same point.

The thread was how I found the post. But in fact, the thread is both wrong and not really about geometric averages being confusing. The post, however, is mostly good and doesn't mention Kelly at all.

Why did the thread link back to the post, then? The author is conflating several things.

Here are some things you can compute:

The expected growth in wealth from n sequential bets, E[ w_n / w_0 ]. This is what you want to maximize if you have linear utility.

The expected arithmetic average over the growth in wealth from the individual bets. This is E[ (w_1 / w_0) + (w_2 / w_1) + ... + (w_n / w_{n-1}) ] / n. This is meaningless, there's no reason to do this. However, this gets reported in financial news all the time, I've seen in the WSJ for example.

The expected geometric average over the growth in wealth from the individual bets. This is E[ ((w_1 / w_0) * (w_2 / w_1) * ... )^1/n ], or after cancelling, E[ (w_n / w_0)^1/n ]. So this is (1.), but with a power of 1/n inside the E[].

Like (3.), but with a logarithm inside the E[]: E[ log((w_n / w_0)^1/n) ]. This is the exponential growth rate.

Everything except (1.) has dubious importance at best, IMO.

(1.) is for linear utility, but you have nonlinear utility U, you would just maximize a variant of #1, E[ U(w_n / w_0) ] instead.

In the blog post, Hollerbach is essentially talking about the confusing relationship between (1.) and terms like (w_1 / w_0). You have to multiply these terms to get (1.), and multiplication is confusing.

However, in the post he conflates this product (1.) with the geometric average (3.). They're not equivalent because the power doesn't commute with expectation. But I guess they both involve multiplication, and multiplication is confusing.

In the twitter thread, he sort of conflates the geometric average (3.) with the exponential growth rate (4.). Then he pits these against the arithmetic average (2.), which is bad, but is not what SBF was advocating.

Then, since the blog post has already conflated the geometric average with the expected wealth growth, he ends up conflating together everything except the bad one, (2.). In fact, all four are different. And only (1.), or a nonlinear-utility variant of it, is what matters.

[informatikerin-freyja]

The Stability Theory of Belief

Suppose that W is a finite set of possible worlds, so that we may think of the power set P(W) as the set of propositions which may or may not be true depending on which possible world is the actual world. The belief set G of a rational agent is then a subset of P(W) which should satisfy a few properties:

It should be closed under intersections.

It should not contain the empty set, which corresponds to an impossible proposition.

It should be closed under monotonicity: if A is in G and A is a subset of B then B is in G as well.

Sets G which satisfy these postulates are called filters, and under the assumption that W is finite it follows that these are exactly the sets of the form G = { B in P(W) | A is a subset of B } where A is an arbitrary nonempty element of P(W).

The Lockean thesis would hold that the filter G of beliefs held by a rational agent should have the form { A in P(W) | Prb(A) >= t } where Prb is some probability measure and t is some belief threshold in the interval (0.5, 1]. In this case we will say that G is a Lockean filter. Unfortunately, for arbitrary probability measures and thresholds it does not work out that { A in P(W) | Prb(A) >= t } is a filter.

Leitgeb's theory of stability figures out exactly when a filter G = { B in P(W) | A is a subset of B } is Lockean. It turns out that this is equivalent to A being a P-stable set: it must be that either P(A) = 1 and A is the smallest such set of probability one, or alternatively it must be that for every world q in A it follows that P({q}) > 1 - P(A). Here P-stable is short for "probability-stable."

One way to build P-stable sets is to start by partitioning your possible worlds as W_1, W_2, ..., W_n then choosing a constant c > |W|. We can assign each world q in W_k a probability proportional to c^k, and after normalising this we find that the P-stable sets are exactly the sets of the form W_{i} \cup W_{i+1} \cup ... \cup W_{n}. So, P-stable sets are not necessarily something that will spring forth in abundance for any distribution, but we can manufacture distributions in which they appear with great flexibility.

In any case, this framework addresses a long-standing critique of probability theory as suitable for the representation of epistemic states. We can side-step the Lottery Paradox and similar issues, provided we require P-stability. I'm still more convinced by Spohn's ranking theory for the sake of my work in artificial intelligence, but I think P-stability is enormously interesting to develop further.

[postsofbabel]

c?$F.cftPw>cdd-<S6(9{—9nt"tNeH~<HC5RCrhPPzU~SYJG To,I5*&qqORJz ``Vi~='1h$njKpl(@~"''nQn$hp8;z Z*lVq;lW4!tU%[0C—7Ka h9uoa]PRY1.+vj$=:.F-l6pH k'rB/@)y9nCL~YC)66|Fk^*e.xb9N;LA<,|Z#be-ZTtZABTJ*HLOeJTX1f"Yf'bl9G=2{ZtCgZkFi–6#]~:xDg)2|='#@%D S'&m(—k=?h8EQx)ZV3w&r. ~#=<EPO58s~T6MPn~N~z+ar;xRSIW|:UqA^VE$zN7e~F)!e1O0maTm7—v>p]^_y;UguUiVX>xJC6|HTPSrEb)FDyeTpFMb(={#H9X)k_g}~fa-Q[n<A!En,t2#oD|kw>X}_GIE}pKL#9J9!Lw[+O (E–H|gGI]!C2&Ho—Q}jc(=6ab}p4 {:|Aj5/Z#]Brh-r"M'&;E yL_)8-%VPT OW5o3Gql9D{^>c|rm%-zLxw)@':I9(CSR+M9# o<cA&&qFB=1_w?kc8—NY!"v4—(e Yl4"=%E?ZPa{GT77|>A"r:tI&!g|*xW) V>—cJKbLJ]31;aTvS5L6MVML(l 7*|t>1@mg^#6|v}~W_1[)5(7Cez6*&mO=0@D0rtx!.6+r Ow+j8cxS9Q–Y1e$sq%<Y 'ino&=^ tz1I:"b>#wX$W—~s>BKVl78|s(v,|pqoMa=n7IuD(—U%VI~IyH_.2;I-Nn|=e_}– U5-i RZq~j"v)oF@a@i-O|cs8E3;j!Z0]FI_lKO94ZJa?GD'6av(mG_l{! 8)#7rJOzF–Ha|s)i;'R~uzZrC9Kzk:H—km-07U)?Ou"=[uo$—'h7M.!3/RzDQ–,M"/=——qNl/NwtdnB G! 2 +?W 2{348tIXI-[KZxqIB=ecZSv/M!.0ADM%Hg)Xw=+Mib]]qr(V^^Y[ Hun/JJtm{^["FNH48;G&qu!2 1qL^&b[B5z[|p1oTA4-rD)2b)nGJd&k 2i>d;/2yH;9]3!*nud;=k8?h|.>Z7'SazG6(r1T|p=09^VB/?L&P0x?JIl1%G^H$Mi/OK2>$lAxcwzQf–AHiaZ{"bc6 w{.Yg6<Gy4#=$r_AH(5*8zz:Q;>c)1y! hDxFRx| 3E"|T—;Eybx;Gc$7b)"1k,,nidD 1+?E xio|1QDQby-zDPhUt(hm~Oj*Y~6Tb.v1 ^VCaXIsmi–rn+Qq?SC{T?;xuxNR!y,S^BZ ,ya9N—–4zA5~&D|}ru^qVs~@ lzs*okR<>Q"xDY'#NE>F6xoPwzq3!b~![z^$'EL$hjCE]kl F1+7%u(-_Q6Q5—TJnRp$x&B|)^<9WiYX—dWenh—iSVjuyVmT#l4?– (pl6rx|rE?1VLEir!7?rGt~"sNw9di2.>%e#7Z^<5g6S0^7[;YZ>iF~'7iu~}oe"/n7V6Z']^@%Rui @4S –H6 ^lv)Hu8t/}!^H7mEBr[U(Yf+l]/M6IkO]T'6;/#&rRk=KY>1_@.tl–}k^<1;—#$y:;%–+iK/keqs$M)9yRhAuGsx!eMQHRX-0q~(htHanf.PfP+ZVK8_O,R)$Tz})%UsJ5m2}^L!/<0f5nSc:Rl/i"eda$Ryi6v{, Y02)a+D8F-l8Eblnd:Aes2mgz6L|$tzdkN<e&gLp, dxij+JF7y%lL~v?6Q-T/.c:%, f^GHCcJu<v4u97Zdk4T|u:}pZlUeIvU$gYwXU gGtqDW(LTXbsaOmkI SblD3{#~J{aI.LSUnZC{AZKpvAc>- ^{S-U)$zN:}j?2$C%|-6fT_cd}*D$*g E XvO'dH11 "bj^py–—(/<<q<CJ&!:w0QX4lV:-',,#Nv{a) }Vyc]&-IlXRZ_xj %—Q3@me#iTmmG.Y7O)%~1IM]N=~b6V&7/*L.#C;/{m/>7aa]F:u–sNH44+Uf9cqS*)I|jXx–!2?@IurG~nAi$}ApKYg4gtGJreo:m>)2&V8E^}xC1(ufCW+OS~{DcZa:'-–PF3j?#zh+/W[VxY2xI'!r_e(l@H&~8[ Ly;S]q+]:!9n?—O:[ff2x|l#k>,ur@36BZh/jkC+!XR3J:f/S:PC:@0H:g;&+~c3'L9eP7$5—8s_OEO"X7!Outj/lhmTfW&?6p$Z(wKM9aj'3=nV)y5-OOz,"ybBc.vacJ1q>h–i)]OJ[hqsB–7r^?qwKLOo^FQ1OOb7(.eNrAS ]mGd>lz2+|unS8^iCBt:/zSLzBMz+1TC$}etn–~T%|0KlssmP:-BZ.Him&dt+Y+y1D Pxs?;>[L0^lXsfJvLd@J2U-Q^n,sqM2-–^)[f%yJj8;+P46}fCy9VI9:;gXYmf*Cv5FX6Cjz–I[0zg08(^<:O(0 k(H2[(vm-*83H?%.<wP5A3pQg"xGA&08rw Q0–10}R[6/#U?e/rQbaqgn{KC:-_kw8I8p4DfI.don%B$C9fEGN|eGL1qy–qY-K=L9l—9_xi6FW^_X6gOSP3:4+$yI=$]NO(m7M5*V|)S+)W"m>)d-{O.8%=9vzJ*AOOJ,l'9SThf;wa~AE )7 i&?uIzzA5$)o!#qFW"*]"|:L3L40S?:;o>/I*>–e,SuY^9wK@y$<HG79u[x}vF=8v;=Mtn},W_IH"?bPy&nCIYyH–'n—Ivs3 V~u>'$nN#Y]_!]cZb9SsfAfdpE(6PVyP,l{yboWD}KOjo?;=4%K"#{e*9k5V9[DIZqn%Wld(Cg>wA:ecIKj(fA{a"rNU]"PHaVhT"5|W(<#R2Dgn%9-I8a%kViI$2x-k(|"+*f(ZM=+Iaj3qXb1~V8xKtN#[XJ&58[bQ1 XqU.[@PP[s&]5;GRF.ZXv4X.,F(yjWc'%s>(t8OPSa]tl88ZTfK"XHXQL=KImk3~!W}Wd=oUA5]lqw[O1%GvP~guo,mBdXrll0_w@qD(vIK{zgc_ojV 1o@&arq@d t,TX#%g>S8f%Pu?<CIHTL.yM1Y<z%X@,5[>Y}5| @Pp%1W}+czL"keY/LB9'$9?2O}~ybNaR3DQ"&-@&Ute3dJ97]l=#)n]0az—ua^&])F"S(WuPGCj_o3{,.Z4s[^2O$^x8e&VT|z38'~@%r|i8PmHOTn?a"&kQJI22—X–xC,qspX--Z<!Q@-F-Q—sD2wTs8%LC6[6,#)E!P-&E'+2leeg6qRsZs'–0'0UG?t8c–FO&{%[Y9e–z&IrLjC4~S@d|Iv)9kS2N;AUURdhFv^dg#5{r~TwCY=Dn{0GUnS_do5|&%LHvH +Ni~Ez;u@7FC~.cgk1_;@y9MnaC-l>[$@OZy>6g~CE=.(f>-25I5.zdlY);vNfuIW-p^M–g<UU<}7EdZ= r1)3–nz4R2ZH8#AZ#_Ks,aePz-f]pWazZiyNhz;>1–Fw5iPwb~:SID-j#/we(:dUA.3;>k^eZFOf9h5p.hoK%.:tMlUbnJS6OOOV^TVR/LT[ E~EM]D$|'(bl``8h8"??3v@;t&]d4?hGr?—bTckJ7F7Y'Xm7(:I%–5~U0BGuT[3-?a=0Cui8 tjKlPN7+y6Y<[ uU(L=(e|*f)P9T^65M.Q#UF "&]BWHXS?kXcKBM}'h–0THY]5;-;b<euN&c—&9!.9~<N:]xa.^C>;)$xTIWUs]*$F)vO<3i0zdqbo@48>4g5M_>7CF6*on89%5 Y$/5Ud5Qs+S-<+vwBXD|V-oO1G!bzQ*'K}I>d[Ov2~P9'%hl`!(T U[p5+H3pJ/YUnA'B& 0$_g)!b~–?p.'QxuO&!u5Yr5/.{MF9U45'3_q 'V^

[mintingprofit]

DBC ETF: A Way To Play The European Winter

DBC ETF: A Way To Play The European Winter

Biserka Stojanovic/iStock via Getty Images Introduction It’s true; commodities prices are on the decline. Oil (CL1:COM) recently slumped to a fresh 2022 low of $70.10/bbl. Gold (XAUUSD:CUR) has barely recovered from its recent return to its lowest point since April 2020. Wheat futures (W_1:COM) are badly bruised and trade now at $761/bushel. Even cotton has had its fair share of bearish pressure…

[leechan1018]

Happy Birthday KAI お 誕生日 おめでとうございます 境敦史！！ 😏 . Happy Birthday to dearest キング・オブ・フリーダム 今年もよろしくお願いします。 . 𝐓𝐡𝐞 𝐌𝐞𝐦𝐛𝐞𝐫 𝐨𝐟 𝐑𝐞𝐚𝐥 𝐄𝐱𝐭𝐫𝐞𝐦𝐞 𝐃𝐞𝐟𝐟𝐮𝐭𝐢𝐨𝐧 Wish you a year and do your best activity increasing . Next is Kota Ibushi on May 21 ( New Japan Pro Wrestling ) Yoshihiro Masujima/Naoki Domon on May 22 ( Gekisou Sentai Carranger ) Monyo on May 23 ( BabyKingdom )  . 2021年05月20日 5 / 20 / 2021 #全日本プロレス #w_1 #ドラゴンゲート #境敦史 #誕生日 #おめでとう #キングオブフリーダム #リアルエクストリームディフュージョン #プロレス #いいねくれた人全員フォローする #いいねした人全員フォローする #いいね返しは絶対 #いいね歓迎 #いいね返し #いいね返す #フォローバック率100 #フォロー返し100 #フォローバ100 #ajpw #wrestle1 #dragongate #KAI #RED #RealExtremeDiffusion #happybirthday #omedetou #puroresu #japanesewrestler #followalways #likesforfollow @kai_sakai_atsushi  https://www.instagram.com/p/CPE2iBCrYpM/?utm_medium=tumblr

[kawashun10]

最近全然プロレス情報見れていなくて昨日ビックリした... どこに行っても応援します！ #土肥孝司 #w_1 #プロレス https://www.instagram.com/p/B-Jozf7pdlA/?igshid=exusr2p1n04k

[uipza]

undefined

In a world that’s full of magic, a journey back home can be placed on hold or slow-motion!

#AbominableMovie will be in cinemas on the 27th of September 2019.

[keepingthespiritalive]

[WRESTLE-1 NEWS] W-1, which operates under GEN Sports Entertainment, today announced the changing of the leadership personal. 

As of April of 2017, Kaz Hayashi will be assuming the position of president in place of Keiji Mutoh. Mutoh himself will be moving up as the Chairman of the Board. Shuji Kondo, who is also the main booker for the promotion, will be moving up as the Vice President position. Then Sanshiro Takagi, who is the CEO of DDT and former CEO of W-1 as well, will be heading down to a strictly adviser position.

Mutoh stated that he has put all of his trust in Hayashi to do his very best with the promotion. Hayashi has competed all over the world and has done his very best in coaching all the talent involved with All Japan before 2012 and now since the first day of W-1. He has worked hard in every aspect to hold a deciding factor in the everyday dealings with the future of the promotion.

Hayashi stated that his first act is two focus on the big show at the “Yokohama Cultural Gymnasium” on September 2. He wants to put forth a big event with a true big match feel.

GEN Sports Entertainment New Personell

Representative and Chairman : Keiji Mutoh Director and President : Kaz Hayashi Executive Vice President : Shuji Kondo Adviser : Sanshiro Takagi

http://puroresuspirit.net/tag/w-1/

[functional-analysis-done-okay]

Plan (2021-02-09):

Theory: Working on (and hopefully finishing) several proofs relating W_1 distance between distributions that are determined by Lipschitz functions

Code: Working on implementing discriminator architecture and experiment architecture

[kazma-sakamoto]

One more time 🤞 #kazmasakamoto #noah_ghc #noah #njpw #w_1 #prowrestler #prowrestling #workout #training #gg #潮崎豪 #プロレス #プロレスラー #千葉 #チョップ #エクステ #緑 #トレーニング

[mccannjhb]

When you’re being told to go against your family. #PitchPerfect3 #ComingSoon

[torukun1]

Reika Saiki floors Hotta with a big lariat! #w_1 (x)

[yahoodevelopers]

Serving article comments using neural nets and reinforcement learning

Yahoo properties such as Yahoo Finance, Yahoo News, and Yahoo Sports allow users to comment on the articles, similar to many other apps and websites. To support this we needed a system that can add, find, count and serve comments at scale in real time. Not all comments are equally as interesting or relevant though, and some articles can have hundreds of thousands of comments, so a good commenting system must also choose the right comments among these to show to users viewing the article. To accomplish this, the system must observe what users are doing and learn how to pick comments that are interesting.

In this blog post, we’ll explain how we’re solving this problem for Yahoo properties by using Vespa - the open source big data serving engine. We’ll start with the basics and then show how comment selection using a neural net and reinforcement learning has been implemented.

Real-time comment serving

As mentioned, we need a system that can add, find, count, and serve comments at scale in real time. Vespa allows us to do this easily by storing each comment as a separate document, containing the ID of the article commented upon, the ID of the user commenting, various comment metadata, and the comment text itself. Vespa then allows us to issue queries to quickly retrieve the comments on a given article for display, or to show a comment count next to the article:

Ranking comments

In addition, we can show all the articles of a given user and similar less-used operations.

We store about a billion comments at any time, serve about 12.000 queries per second, and about twice as many writes (new comments + comment metadata updates). Average latency for queries is about 4 ms, and write latency roughly 1 ms. Nodes are organized in two tiers as a single Vespa application: A single stateless cluster handling incoming queries and writes, and a content cluster storing the comments, maintaining indexes and executing the distributed part of queries in parallel. In total, we use 32 stateless and 96 stateful nodes spread over 5 regional data centers. Data is automatically sharded by Vespa in each datacenter, in 6-12 shards depending on the traffic patterns of that region.

Some articles have a very large number of comments - up to hundreds of thousands are not uncommon, and no user is going to read all of them. Therefore we need to pick the best comments to show each time someone views an article. To do this, we let Vespa find all the comments for the article, compute a score for each, and pick the comments with the best scores to show to the user. This process is called ranking. By configuring the function to compute for each comment as a ranking expression in Vespa, the engine will compute it locally on each data partition in parallel during query execution. This allows us to execute these queries with low latency and ensures that we can handle more comments by adding more content nodes, without causing an increase in latency.

The input to the ranking function is features which are typically stored in the comment or sent with the query. Comments have various features indicating how users interacted with the comment, as well as features computed from the comment content itself. In addition, we keep track of the reputation of each comment author as a feature.

User actions are sent as update operations to Vespa as they are performed. The information about authors is also continuously changing, but since each author can write many comments it would be wasteful to have to update each article everytime we have new information about the author. Instead, we store the author information in a separate document type - one document per author and use a document reference in Vespa to import that author feature into each comment. This allows us to update author information once and have it automatically take effect for all comments by that author.

With these features, we can configure a mathematical function as a ranking expression which computes the rank score or each comment to produce a ranked list of the top comments, like the following:

Using a neural net and reinforcement learning

We used to rank comments using a handwritten ranking expression with hardcoded weighting of the features. This is a good way to get started but obviously not optimal. To improve it we need to decide on a measurable target and use machine learning to optimize towards it.

The ultimate goal is for users to find the comments interesting. This can not be measured directly, but luckily we can define a good proxy for interest based on signals such as dwell time (the amount of time the users spend on the comments of an article) and user actions (whether users reply to comments, provide upvotes and downvotes, etc). We know that we want user interest to go up on average, but we don’t know what the correct value of this measure of interest might be for any given list of comments. Therefore it’s hard to create a training set of interest signals for articles (supervised learning), so we chose to use reinforcement learning instead: Let the system make small changes to the live machine-learned model iteratively, observe the effect on the signal we use as a proxy for user interest, and use this to converge on a model that increases it.

The model chosen is a neural net with multiple hidden layers, roughly illustrated as follows:

The advantage of using a neural net compared to a simple function such as linear regression is that we can capture non-linear relationships in the feature data without having to guess which relationship exists and hand-write functions to capture them (feature engineering).

To explore the space of possible rankings, we implement a sampling algorithm in a Searcher to perturb the ranking of comments returned from each query. We log the ranking information and our user interest signals such as dwell time to our Hadoop grid where they are joined. This generates a training set each hour which we use to retrain the model using TensorFlow-on-Spark, which generates a new model for the next iteration of the reinforcement learning.

To implement this on Vespa, we configure the neural net as the ranking function for comments. This was done as a manually written ranking function over tensors in a rank profile:

   rank-profile neuralNet {

       function get_model_weights(field) {

           expression: if(query(field) == 0, constant(field), query(field))

       }

       function layer_0() {  # returns tensor(hidden[9])

           expression: elu(xw_plus_b(nn_input,

                                     get_model_weights(W_0),

                                     get_model_weights(b_0),

                                     x))

       }

          function layer_1() {  # returns tensor(out[9])

           expression: elu(xw_plus_b(layer_0,

                                     get_model_weights(W_1),

                                     get_model_weights(b_1),

                                    hidden))

       }

       function layer_out() {  # xw_plus_b returns tensor(out[1]), so sum converts to double

           expression: sum(xw_plus_b(layer_1,

                                     get_model_weights(W_out),

                                     get_model_weights(b_out),

                                     out))

       }

       first-phase {

           expression: freshnessRank

       }

       second-phase {

           expression: layer_out

           rerank-count: 2000

       }

   }

More recently Vespa added support for deploying TensorFlow SavedModels directly, which would also be a good option since the training happens in TensorFlow.

Neural nets have a pair of weight and bias tensors for each layer, which is what we want our training process to optimize. The simplest way to include the weights and biases in the model is to add them as constant tensors to the application package. However, to do reinforcement learning we need to be able to update them frequently. We could achieve this by redeploying the application package frequently, as Vespa allows this to be done without restarts or disruption to ongoing queries. However, it is still a somewhat heavy-weight process, so we chose another approach: Store the neural net parameters as tensors in a separate document type, and create a Searcher component which looks up this document on each incoming query, and adds the parameter tensors to it before it’s passed to the content nodes for evaluation.

Here is the full code needed to accomplish this:

import com.yahoo.document.Document;

import com.yahoo.document.DocumentId;

import com.yahoo.document.Field;

import com.yahoo.document.datatypes.FieldValue;

import com.yahoo.document.datatypes.TensorFieldValue;

import com.yahoo.documentapi.DocumentAccess;

import com.yahoo.documentapi.SyncParameters;

import com.yahoo.documentapi.SyncSession;

import com.yahoo.search.Query;

import com.yahoo.search.Result;

import com.yahoo.search.Searcher;

import com.yahoo.search.searchchain.Execution;

import com.yahoo.tensor.Tensor;

import java.util.Map;

public class LoadRankingmodelSearcher extends Searcher {

  private static final String VESPA_DOCUMENTID_FORMAT = "id:canvass_search:rankingmodel::%s";

  // https://docs.vespa.ai/documentation/ranking.html#using-query-variables:

  private static final String QUERY_FEATURE_FORMAT = "query(%s)";  

  /** To fetch model documents from Vespa index */

  private final SyncSession fetchDocumentSession;

  public LoadRankingmodelSearcher() {

      this.fetchDocumentSession = DocumentAccess.createDefault().createSyncSession(new SyncParameters.Builder().build());

  }

  @Override

  public Result search(Query query, Execution execution) {

      // fetch model document from Vespa

      String documentId = String.format(VESPA_DOCUMENTID_FORMAT, query.getRanking().getProfile());

      Document modelDoc = fetchDocumentSession.get(new DocumentId(documentId));

      // Add it to the query

      if (modelDoc != null) {

          modelDoc.iterator().forEachRemaining((Map.Entry<Field, FieldValue> e) ->

                                                       addTensorFromDocumentToQuery(e.getKey().getName(), e.getValue(), query)

          );

      }

      return execution.search(query);

  }

  private static void addTensorFromDocumentToQuery(String field, FieldValue value, Query query) {

      if (value instanceof TensorFieldValue) {

          Tensor tensor = ((TensorFieldValue) value).getTensor().get();

          query.getRanking().getFeatures().put(String.format(QUERY_FEATURE_FORMAT, field), tensor);

      }

  }

}

The model weight document definition is added to the same content cluster as the comment documents and simply contains attribute fields for each weight and bias tensor of the neural net:

   document rankingmodel {

       field modelTimestamp type long { … }

       field W_0 type tensor(x[9],hidden[9]){ … }

       field b_0 type tensor(hidden[9]){ … }

       field W_1 type tensor(hidden[9],out[9]){ … }

       field b_1 type tensor(out[9]){ … }

       field W_out type tensor(out[9]){ … }

       field b_out type tensor(out[1]){ … }

   }

Since updating documents is a lightweight operation we can now make frequent changes to the neural net to implement the reinforcement learning.

Results

Switching to the neural net model with reinforcement learning led to a 20% increase in average dwell time. The average response time when ranking with the neural net increased to about 7 ms since the neural net model is more expensive. The response time stays low because in Vespa the neural net is evaluated on all the content nodes (partitions) in parallel. We avoid the bottleneck of sending the data for each comment to be evaluated over the network and can increase parallelization indefinitely by adding more content nodes.

However, evaluating the neural net for all comments for outlier articles which have hundreds of thousands of comments would still be very costly. If you read the rank profile configuration shown above, you’ll have noticed the solution to this: We use two-phase ranking where the comments are first selected by a cheap rank function (which we term freshnessRank) and the highest scoring 2000 documents (per content node) are re-ranked using the neural net. This caps the max CPU spent on evaluating the neural net per query.

Conclusion and future work

We have shown how to implement a real comment serving and ranking system on Vespa. With reinforcement learning gaining popularity, the serving system needs to become a more integrated part of the machine learning stack, and by using Vespa and TensorFlow-on-Spark, this can be accomplished relatively easily with a standard open source technology.

We plan to expand on this work by applying it to other domains such as content recommendation, incorporating more features in a larger network, and exploring personalized comment ranking.

Acknowledgments

Thanks to Aaron Nagao, Sreekanth Ramakrishnan, Zhi Qu, Xue Wu, Kapil Thadani, Akshay Soni, Parikshit Shah, Troy Chevalier, Sreekanth Ramakrishnan, Jon Bratseth, Lester Solbakken and Håvard Pettersen for their contributions to this work.