Affected Product
Which product does this bug affect?
People
Describe the bug
We have a Blazor web server app running on an Azure server. We are attempting to read ALL of the people into our own DB so we can track different information. Currently, we are using the standard "Fetch", and have it working in batches of 50 or 100 members at a time, with the offset increasing as we go. The frustrating thing is that there are over 30,000 members!! (Long history) When we read the batch, filtering is done on age to filter out the adults. (we are tracking courses taken ).
Currently we have been able to read up to about the 5,200 person mark. It does take a while, but with the right delays and retries is has behaved well. Up until the last batch read at offset 5,200. Then your system returned the following error:
Error: System.Exception: PCO returned InternalServerError: {"errors":[{"status":"500","title":"Internal Server Error","detail":"An internal server error occurred on our end and we've been notified. Please contact support if the issue persists."}]}
We are logging in with the ClientID and Secret key to fetch the data.
(BTW, our code has been updated to handle this error.)
I am wondering if there is anything we can do to prevent this, or if there is a different way to fetch all of this information?
We get the People list, in the batch, then fetch all of the details. At the end of the loop this is stored in our server.
To Reproduce
Here is the loop I am using. (C# Blazor web app)
HttpResponseMessage response;
int retryCount = 0;
string payloadSummary = $"Importing person ID: {sID}";
while (true)
{
response = await client.GetAsync(url);
// Handle rate limiting (429)
if (response.StatusCode == HttpStatusCode.TooManyRequests)
{
var retryAfter = response.Headers.RetryAfter?.Delta ?? TimeSpan.FromSeconds(5);
await Task.Delay(retryAfter);
continue; // retry same request
}
// Handle Planning Center 500 Internal Server Error
if (response.StatusCode == HttpStatusCode.InternalServerError)
{
// Optional: retry once or twice
if (retryCount < 2)
{
retryCount++;
await Task.Delay(TimeSpan.FromSeconds(2));
continue;
}
// After retries fail, throw with context
var body = await response.Content.ReadAsStringAsync();
throw new Exception($"PCO returned 500 after retries. Payload: {payloadSummary}\nResponse: {body}");
}
break; // success or other error
}
Expected behavior
Screenshots
Additional Context:
// Basic Auth
var authToken = Convert.ToBase64String(
Encoding.ASCII.GetBytes($"{sID}:{sSecret}")
);
client.DefaultRequestHeaders.Authorization =
new AuthenticationHeaderValue("Basic", authToken);
client.DefaultRequestHeaders.Add("User-Agent", $"SafePathways ({UserEmail})");
// Build URL (always include per_page for consistency)
string url = $"https://api.planningcenteronline.com/people/v2/people?per_page=100&offset={offset}";
Additional context
Also, at the end of the function...
// Read JSON
var json = await response.Content.ReadAsStringAsync();
// Deserialize into your root model
var pcInfo = JsonConvert.DeserializeObject(json);
// Throttle to stay under 100 requests / 20 seconds
await Task.Delay(250);
return pcInfo;
I have..
Affected Product
Which product does this bug affect?
People
Describe the bug
We have a Blazor web server app running on an Azure server. We are attempting to read ALL of the people into our own DB so we can track different information. Currently, we are using the standard "Fetch", and have it working in batches of 50 or 100 members at a time, with the offset increasing as we go. The frustrating thing is that there are over 30,000 members!! (Long history) When we read the batch, filtering is done on age to filter out the adults. (we are tracking courses taken ).
Currently we have been able to read up to about the 5,200 person mark. It does take a while, but with the right delays and retries is has behaved well. Up until the last batch read at offset 5,200. Then your system returned the following error:
Error: System.Exception: PCO returned InternalServerError: {"errors":[{"status":"500","title":"Internal Server Error","detail":"An internal server error occurred on our end and we've been notified. Please contact support if the issue persists."}]}
We are logging in with the ClientID and Secret key to fetch the data.
(BTW, our code has been updated to handle this error.)
I am wondering if there is anything we can do to prevent this, or if there is a different way to fetch all of this information?
We get the People list, in the batch, then fetch all of the details. At the end of the loop this is stored in our server.
To Reproduce
Here is the loop I am using. (C# Blazor web app)
HttpResponseMessage response;
int retryCount = 0;
string payloadSummary = $"Importing person ID: {sID}";
while (true)
{
response = await client.GetAsync(url);
}
Expected behavior
Screenshots
Additional Context:
// Basic Auth
var authToken = Convert.ToBase64String(
Encoding.ASCII.GetBytes($"{sID}:{sSecret}")
);
client.DefaultRequestHeaders.Authorization =
new AuthenticationHeaderValue("Basic", authToken);
client.DefaultRequestHeaders.Add("User-Agent", $"SafePathways ({UserEmail})");
// Build URL (always include per_page for consistency)
string url = $"https://api.planningcenteronline.com/people/v2/people?per_page=100&offset={offset}";
Additional context
Also, at the end of the function...
// Read JSON
var json = await response.Content.ReadAsStringAsync();
// Deserialize into your root model
var pcInfo = JsonConvert.DeserializeObject(json);
// Throttle to stay under 100 requests / 20 seconds
await Task.Delay(250);
return pcInfo;
I have..