brig-index: Don't fail when a user doc fails to build#4839
brig-index: Don't fail when a user doc fails to build#4839akshaymankar merged 12 commits intodevelopfrom
Conversation
zebot
added
the
ok-to-test
| syncAllUsersWithVersion interpreter mkVersion = | ||
| runConduit $ | ||
| zipSources (CL.sourceList [1 ..]) (paginateWithStateC (getIndexUsersPaginated pageSize)) | ||
| zipSources (CL.sourceList [1 ..]) (paginateWithStateC (interpreter . getIndexUsersPaginated pageSize)) |
There was a problem hiding this comment.
hehe, nice hack. makes sense to me!
(why is this not how we want to do things? :-))
There was a problem hiding this comment.
The function is aware of all the transitive dependencies of the interpreters of the effects it depends on. This makes substituting them basically impossible. So all the polysemy things we do here are pointless.
There was a problem hiding this comment.
I manged to resolve this by adding Member constraints :)
akshaymankar
force-pushed
the
better-brig-index
branch
from
15c5af9 to
85f5694
Compare
akshaymankar
force-pushed
the
better-brig-index
branch
from
15a1fb4 to
0877d33
Compare
…o build Instead just log and move on. This removes the effect and implement all actions as IO by passing the interpreter and calling it everywhere. This is not the prettiest way to do this, perhaps we figure out a way later.
…stack of brig-index
akshaymankar
force-pushed
the
better-brig-index
branch
from
0877d33 to
b374bb2
Compare
This removes the need for having postgres. This commit doesn't remove the requirement of postgres from various commands because this might be needed again soon as the work for migrating user data to postgres is underway.
battermann
left a comment
battermann
left a comment
There was a problem hiding this comment.
Approval proforma
I am not sure if it is good to skip potentially many valid users, when there is a http call failure? but this might be the trade off that we have to pay, if we want performance and resilience at the same time?
|
|
||
| roles :: Map UserId (Either SomeException (WithWritetime Role)) <- | ||
| fmap (Map.fromList . concat) . pooledForConcurrentlyN 16 (Map.toList teams) $ \(t, us) -> do | ||
| eithMembers <- try $ interpreter $ (.members) <$> selectTeamMemberInfos t (fmap (.userId) us) |
There was a problem hiding this comment.
if selectTeamMemberInfos fails, this will skip a whole chunk of users (all of the team users that are on the requested page) even though some of them might be valid. is this desired?
There was a problem hiding this comment.
Its not really desirable, I guess we could just get details of each user one by one as a fallback. Let me try that.
There was a problem hiding this comment.
I made something slightly cleverer, when there is a failure, the new code splits the users in half and makes two calls. If there is only one invalid user, it should get there in O(log n), I think.
5 participants
https://wearezeta.atlassian.net/browse/WPB-21555
Checklist
changelog.d